Update packages from crontab

Warning: Doing automatic updates from cron is strongly discouraged. It is likely to leave your machine in a broken and unbootable state. If this breaks your machine, do not hold anyone but yourself responsible. You have been warned.

Do not try this at home!

So, proceed only if you have balls of steel or you were intending to crash your machine anyway. If so, you might as well do it the "right way".

  1. First, you (obviously!) need to install cron itself. Do that first.
  2. It is highly recommended to also install a mail transfer agent, such as Postfix, to send you notifications if pacman fails.
  3. Run as root: crontab -e
  4. Copy-paste this to your crontab:
MAILTO=your@email
LOGFILE=/var/log/cron-pacman.log

# 1. minute (0-59)
# |   2. hour (0-23)
# |   |   3. day of month (1-31)
# |   |   |   4. month (1-12)
# |   |   |   |   5. day of week (0-7: 0 or 7 is Sun, or use names)
# |   |   |   |   |   6. commandline
# |   |   |   |   |   |
#min hr  dom mon dow command
00   13   *   *   *  . /etc/profile && (echo; date; yes |pacman -Syuq) &>>$LOGFILE || (echo 'pacman failed!'; tail $LOGFILE; false)
To check every */time you must use e.g. */2 to check every 2 min or any other time value.

If you want to automatically reboot your computer upon a successful upgrade, append '&& reboot' to the above line.

Do try this at home!

Instead of using `pacman -Syuq` above, use `pacman -Syuwq`. The '-w' will cause pacman to "retrieve all packages from the server, but do not install/upgrade anything." While you will still have to manually update your system, you won't have to wait (as long) for packages to download while doing so.

Manual set up of auto-update

Note: The sections below are mostly focused on setting up Linux on computer for the home user.
Warning: You must make backup of pacman database before giving computer to user.

If you want to set up Arch Linux for home user and still keep system up to date you will need to make work in background without distraction of user.

Note:
  • If you will need to run other scripts from inside of script that run by crone then you will need to use the full path to them, e.g. pacman -S --needed --noconfirm $(/path/to/script/ShortListUpdates.sh) or they will not run if you will use only their name without full path.
  • Use the pacman-db-upgrade command only in the end after installing packages because it may leave the /var/lib/pacman/db.lck lock file that will prevent run of the next pacman command. It happens only when scrip is running by cron but not when you start it manually from console.

Update notifier

There are many programs such as zenity that can show GUI messages from program or scripts that runs in console. In this example you will see a dialog with list with file names and size for each of them that need to be downloaded for installation. By clicking Yes it will return exit code 0 respective 1 for No that will be placed in the $? variable.

$ zenity --question --text="$(pacman -Qu|awk '//{if(index($0,"[ignored]") == 0 )print $1  }'|pacman -Si -|grep ^[a-Z]| \
 sed 's/ //g'|  awk -F':' '//{ZZ=ZZ+1;XX[ZZ]=$1;SS[ZZ]=$2;}   
 END{AA=ZZ/18;for(i=1;i <= AA;i++){print XX[i*18-16]": "SS[18*i-16]" | "XX[i*18-4]": "SS[i*18-4];}}' )"

For a long list of updates is better to use this example and zenity can also read data from stdin.

$ pacman -Qu |awk '//{if(index($0,"[ignored]") == 0 )print $1  }'|pacman -Si -|grep ^[a-Z]|\
  sed 's/ //g'|awk -F':' '//{ZZ=ZZ+1;XX[ZZ]=$1;SS[ZZ]=$2;}
  END{AA=ZZ/18;for(i=1;i <= AA;i++){print XX[i*18-16]": "SS[18*i-16]" | "XX[i*18-4]": "SS[i*18-4];}}' | zenity --text-info --title=Uppdateringar

Is suitable to use after updates has been downloaded.

This will show progress bar. Depends on amount of repositories you will need to calculate correct step size, you can do it simply by dividing 100 with amount of lines printed by pacman or use LCount=$(grep ^"\[" /etc/pacman.conf -c);echo $((100/LCount)) to calculate and pass it to the awk command.

$ pacman -Sy --noprogressbar |awk '{AA=AA+16.7;system("echo "AA)}' | zenity --progress --no-cancel

In comparison to zenity the program xmessage from xorg-xmessage package shows much smaller windows but doesn't support UTF and has some more limitations but here we will not discuss them, it is just a perfect light notifier.

...........
...........
STARTit=$(date '+%M-%S')
xmessage -timeout 2 "Starting downloading of updates" & disown
...........
...........
ENDit="$(date '+%M-%S')"
xmessage -timeout 3 "${STARTit}"' * '"${ENDit}" & disown
...........
...........

See also: How to show a message box from a bash script in Linux.

Before running pacman you can also add integrity of database check:

testdb -b "/var/lib/pacman/"
   if [ ${?} != "0" ];then 
    zenity --warning --text="Databasfilen är skadat. Kontakta Admin!";
    exit 1;
   fi

Download updates

The easiest way of creating a short list of updates is by creating the script that will print them to stdout, e.g.

ShortListUpdates.sh
#!/bin/bash
pacman -Qq | \
grep -e smplayer \
-e smtube \
-e ^jre \
-e ^gst \
-e firefox | \
grep -v -e 'to_ignore_package1' -e 'to_ignore_package2' -e 'to_ignore_package3'
#Files from a group to add
pacman -Qqg kde | grep -v -e 'to_ignore_package1' -e 'to_ignore_package2' -e 'to_ignore_package3'

Good to download updates after few hours of idle and limit it only to a day time and make similar just that will download and possible also install updates at night time if computer is powered on. To speed up update process at night and lower possible damage add some of packages to ignore list, such as fonts, kernel and mkinitcpio or any other that you think is not at all necessary to be updated. But still you can make a script that user can manually run to update all packages, e.g. pacman -Qqen | pacman -S --needed --noconfirm - but do not forget to add safety check of battery status if it is a laptop.

Steps for downloading process

  1. Check if stop mark after download exist, e.g. if [ ! -f /tmp/.downloaded_yes ];then echo Is OK to download;else echo Already downloaded;fi
  2. Check if computer is idle and how long time. Utilities xprintidle[1] for X and command w for tty.
  3. Check if computer is connected with cable or how much battery is charged. You will need to install upower.
    #!/bin/bash
    
    MAXpower="60"
    
    ST=($(upower -i "$(upower -e | grep 'BAT')" | grep -e "state" -e percentage | awk '{print $2}' | sed 's/%//g'))
    if [ ! -z ${ST[0]} ];then
     if [ ${ST[1]} -gt ${MAXpower} ]  || [ ${ST[0]} == 'charging' ] || [ ${ST[0]} == 'fully-charged' ]; then
      echo "OK"
       else
      echo "Fail"
     fi
      else 
     echo "OK"
    fi
  4. Check type of connection to prevent downloading due huge amount of data if computer using 3G or other PPP connection. Command ifconfig | grep ^ppp -c.
  5. Check if is online ping -c 1 8.8.8.8 ; if [ "${?}" != "0" ]; then echo bad; else echo good; fi
  6. Make a short list of the most important updates, e.g. for average internet user will be good to have a web-browser and its plugins,extensions, certifications and SSL componets and some of media players that you configured as default will need to be downloaded.
    Note: Don't make list too big for daily downloads and do not start installation directly after them were downloaded to avoid possible conflicts.
  7. Create a stop mark to prevent multiple update checks after download were finished, e.g. echo downloaded > /tmp/.downloaded_yes.
Note: The stop mark can be also created after all checks are passed in case if check of idle time is too low and download time might take more time depends on the connection and total size of updates.

Download and install updates from short list

To download updates from the short list you can use this line pacman -Sw --noconfirm --needed $(/full/path/to/ShortListUpdates.sh).

You will need to add only a power check before continue with installation of updates.

...............
...............
yes | pacman -S --noconfirm --needed $(/full/path/to/ShortListUpdates.sh)
systemctl daemon-reload
sleep 2
...............
...............

If you will use a GUI notifier then you may want to use this part in the begin of the update script if updates runs after reboot and user comes to log in screen(Display manager). Be careful with the sleep timeout if too low it might freeze the screen. The notifier will be shown even on the login screen that will probably make user wait until updates are finished or after user logged if you will set sleep timeout too high and user logged in before script started.

#!/bin/bash
sleep 5
if [ "$(pgrep X -c)"  != "0" ]; then
export DISPLAY=:0
else
export DISPLAY=:0
startx "$0"
fi
.........
.........
.........

You may also will make it remove unnecessary locales for spell check in e.g. /usr/include/hunspell, /usr/lib/aspell-0.60 or any other that contains unnecessary files that created after update or install of packages.

In the crone you can schedule installation of updates from a short list on each boot by using in crontab:

@reboot /usr/local/bin/update
Note: You can also make it copy the update-downloading scripts to /tmp to minimize disk usage because the /tmp is using RAM for storage of files.

Download and install at night

If user will leave computer powered on at night then this example could be used to download and install after computer become idle for a some time to be more sure that user will not interrupt.

..............
..............# Battery and connection safety checks
..............
yes | pacman -S --needed $(pacman -Ssq openjdk | grep -v -e doc -e src)
  if [ $(ifconfig | grep ^ppp -c)  != "0" ];then
    pacman -Suw --noconfirm
   else
    pacman -Qq --native  | pacman -Sw --needed -
  fi
yes | pacman -Su
if [ "${?}" != "0"  ];then beep -f 100 -l 1000 && zenity --warning --text="Ett problem har uppstått. Kontakta Admin!"
if [ -f "/var/lib/pacman/db.lck"  ];then zenity --warning --text="Databasfilen är skadat eller används: /var/lib/pacman/db.lck";fi;
exit 1
else
BkupDate=$(date '+%A-day_of_the_week-%u')
pacman -Q --native > /opt/.alt_db_bkup-"${BkupDate}"
date >> ${BkupDate}
systemctl daemon-reload
sleep 2
fi
pacman-db-upgrade
..............
..............
..............

Some of updates can be installed directly if you want make it download new releases of updates and not only updates for existing. If computer will use wireless or cable connection it will download all even packages that are in the ignore list but install only updates that are not in the ignore list. You can also make it download all updates one day and install next day, for that you will need to modify script and schedule days of the week for downloading and another script for installing or use command line variables to switch from downloading to installing.

Update packages from the AUR

First must be created the GPG key and added to the makepkg configuration file.

/etc/makepkg.conf
...............
PKGDEST=/path/to/custom_repo
GPGKEY="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
...............

This can be used as a part in the download scripts and in the end of them.

Warning: Sometimes packages can change their names that will prevent update of the equivalent installed or if dependencies will be changed.
iOK="0"
URLarray=(PKG1 URL1
PKG2 URL2)

arrP=0;arrU=1

while [ "${#URLarray[@]}" -gt "$arrU"  ];do

 wget -nd -nH -m "${URLarray[arrU]}" -P /tmp
  if [ "$?" != "0"   ];then
   pacman -Sw "${URLarray[arrP]}" --needed --noconfirm
else
cd /tmp
runuser -u user_name makepkg
  if [ "$?" == 0 ];then iOK="1";fi
rm /tmp/PKGBUILD
fi  
 arrU=$((arrU+2))
 arrP=$((arrP+2))
done

if [ "$iOK" != "0" ]; then
repo-add -s -n /path/to/custom.db.tar  /path/to/*.xz
fi

An alternative for a more complex downloads when some of the files need to be downloaded from other sources like e.g. github. This should be repeated for each file.

wget -nd -nH -m https://aur.archlinux.org/packages/....../PKGBUILD -P /tmp
wget -nd -nH -m  https://raw.githubusercontent.com/....../filename -P /tmp
cd /tmp
runuser -u babak makepkg
if [ "$?" == 0  ]; then
repo-add -s -n /opt/.AUR/x86_64/custom.db.tar /opt/.AUR/x86_64/*.tar.xz
fi
rm /tmp/PKGBUILD
rm /tmp/filename
Note: An alternative to update packages is to download and unzip binaries if are available for your architecture. But the version of packages will not be registered in the pacman database. The best way if it is possible then download by using a single URL instead of parsing the downloaded file for the URL.

Example how to download and install pepper flash binaries. The script is using p7zip for extraction of the rpm to stdout.

#......... #Check if the destination folders are exist and notifiers if needed and if running as root
#......... #
 wget -m -nd -nH https://dl.google.com/linux/direct/google-chrome-stable_current_"$(uname -m)".rpm -P /path/to/.AUR/chromium-pepper-flash -o /tmp/chromium-pepper-flash.wget-log
   if [ "$(grep '100%' -c /tmp/chromium-pepper-flash.wget-log)" != "0" ];then
   if [ ! -d "/tmp/PepperFlash/" ];then mkdir /tmp/PepperFlash/ --mode=755 -p ;fi
    cd "/tmp/PepperFlash/"
    7z e /path/to/.AUR/chromium-pepper-flash/google-chrome-stable_current_"$(uname -m)".rpm -so | cpio -idvm
   chmod 644 /tmp/PepperFlash/opt/google/chrome/PepperFlash/*
   cp --preserve=all -v /tmp/PepperFlash/opt/google/chrome/PepperFlash/* /usr/lib/PepperFlash/
   rm -fr /tmp/PepperFlash/.
#......... # 
#......... # Use notifier if needed
  fi

Update the mirrorlist file

This can be scheduled to update mirrors once a week or few times in a month only for countries you want

rm  /etc/pacman.d/mirrorlist
for Cnt in China Iran Russia Korea;
 do 
awk -v GG=$Cnt '{if(index($0,GG) != 0)AA=1;if(AA == 1)
   {if( match($0,"#") != "0"){SS=$0;sub("#","",SS);print SS ;}else AA=0} }' \
 /etc/pacman.d/mirrorlist.pacnew >> /etc/pacman.d/mirrorlist;

done

See also: What to do if pacman-mirrorlist is not installed.

Recovery after failed update or upgrade

If user did upgrade or update that prevents boot into the desktop environment or applications cannot start but user is able to see the display manager then it is good to make a separate environment for the user that is easy to use to try fix those errors on its own. This will be useful for users that has relatively low knowledge about computer specially about Linux. Install openbox or any other equivalent window manager that they can use with the idesk and create own buttons to the programs or better scripts that are configured to make actions and show warnings and the administrator contact information before starting. To prevent any unwanted user to use the "failsafe" environment or some of the recovery tasks you can add to begin of the script or window manager autostart something like:

AllowedUser="JolinTsai";
if [ "$(whomai)" != " ${AllowedUser,,}" ];then zenity --warning --text="You are not the allowed user!
The allowed user is "$AllowedUser;
#openbox --exit
exit 1
fi

You will need to remove all unnecessary entries from "type of session" list, you can do it by removing or better to moving the *.desktop files which contains information about them to the backup folder and create your own with a custom configuration, to make it simply just copy one of the *.desktop files to a file with the name you want and make changes in the Exec and description parts.

Something like this you can use to remove or move files to backup directory.

find /usr/share/apps/kdm/sessions/ -maxdepth 1 -type f \ 
! -name kde-plasma.desktop ! -name lxde.desktop ! -name openbox.desktop ! -name xfce4.desktop \
-exec mv "{}" /usr/share/apps/kdm/sessions/OLD_/ \;

You usually can find them in some of those folders:

/usr/share/config/kdm/sessions
/usr/share/apps/kdm/sessions
/usr/share/xsessions/

You will also need to remove borders from windows to prevent user to close the working window such as xterm if you will use it to show output of commands while they are working. You can do it by using the devilspie.

To get list of the window names for using in the devilspie configuration file you can use the wmctrl utility wmctrl -l | awk '{print substr($0, index($0,$4))}' or when you start devilspie you will see all information that is possible to use in the configuration file.

Window Title: 'name@host:~/.path'; Application Name: 'name@host:~/.path' ; Class: 'XTerm' ; Geometry: 492x350+487+226

The window_name , application_name and window_class can be used to change the window properties.

Example of the devilspie configuration file that you can use for the preferred application

.devilspie/DesktopConsole.ds
(if (is (window_class) "XTerm")
        (begin
(undecorate)
(skip_tasklist)
(above)
(fullscreen)
(maximize)                       
(unpin)                
(skip_pager)       
            )
        )

The XTerm has also a command line to start it in the full screen: xterm -fullscreen.

Tip: To run an application in the full screen makes user unable to click on other buttons on the desktop until task is completed.

Download and fix damaged packages

Tango-emblem-important.png

Tango-emblem-important.png

The factual accuracy of this article or section is disputed.

Reason: The bash script has mistakes in its code. (Discuss)

If some of the applications got broken then it much easier to download it from the website and extract. By reading logs is possible to determinate which of applications is broken.

Example how to download the pacman package or install it if available in the pacman cache.

#!/bin/bash
RepoType=(core extra community)
Arch="x86_64"
PacCache="/var/cache/pacman/pkg"
LogName="down_pkg_log_"
PathPKG="/tmp" 

PacName="pacman"
RepoSelect="0"
#PacName="$1"
#RepoSelect="$2"

PacFind="$(ls -1 ${PacCache} --sort=version|awk -v QQ=$Arch -v EE=$PacName '//{if( match($0,EE"-[0-9]") == "1" )ZZ=$1 }
END{if(index(ZZ,QQ) != 0 )print ZZ}')";

if [ ! -z "$PacFind" ];then
 tar --exclude-ignore='.MTREE' --exclude-ignore='.PKGINFO'  --exclude-ignore='.INSTALL' -C / -xvf "${PacCache}/${PacFind}"
else
 wget --content-disposition -m -nd -nH https://www.archlinux.org/packages/${RepoType[RepoSelect]}/${Arch}/${PacName}/download/ -P ${PathPKG} -o "${PathPKG}/${LogName}"
  if [  "$?" != "0" ];then
 xmessage "No Internet connection is available 
and 
cannot find the $PacName file in the pacman cache."
exit 1
  else
UnTarPKG="$(awk -v XX=${PacName} -F"$PathPKG" '//{AA=$1;if( $2 ~ "/" && index($2,"[") == "0"  ){KK=$2;
sub(".","",KK);split(KK,SS,"xz");print SS[1]"xz" }}' /tmp/down_pkg_log_)"
 tar --exclude-ignore='.MTREE' --exclude-ignore='.PKGINFO'  --exclude-ignore='.INSTALL' -C / -xvf "${PathPKG}/${UnTarPKG}"
  fi
fi

See also: What to do if pacman is broken beyond repair

Note:
  • For some programs is enough to remove their configuration files and they will create new on the next start but other need to be restored from a back to preserve made by you configuration for the user.
  • To prevent user to delete configuration files by accident you can lock folders from browsing with the chmod 300 . or use 733 mask if you will change owner to root or any other admin user that has permission to use sudo command and do chmod 444 file_name for the files like .bash_logout , .bash_profile , .bashrc , .xinitrc or any other in the user home folder that has no need to be modified or removed.