Post recovery tasks - ArchWiki

# Post recovery tasks

Related articles

- File recovery
- Sort images by resolution

## Photorec

### Creation of a file with data for arrays

In this example the xdg-mime is used to gather information about the mime types but the `file --mime-type -b` and `file -i -b` commands does the same output as the `xdg-mime query filetype` command, with more or less details. This script will collect a lot of more additional information about the files into the info-mime-size-db.txt. Put the script in the destination directory that you used in photorec, make it executable and run it or use the `sh -e start-collect-file-info.sh` command to run it.

    start-collect-file-info.sh

    #!/bin/bash
    ScriptsName=$(basename "$0")
    if [ 'XX' != 'XX'"$1" ]; then
    if [ -f "$1"  ]; then
    echo "$1"
    echo "$(file "$1" -F"|"  )'|'$(xdg-mime query filetype "$1")'|'$(du -h "$1" | awk '{print $1}' )" >> info-mime-size-db.txt
    else
    echo The « "$1" » is not a valid file name.
    fi
    else
    find -type f -exec sh -e "./$ScriptsName" "{}" \;
    fi

The script will build a file with pattern path to file/file name | info about the file | mime type | size, here is an example:`./recup_dir.1/f872690288.jpg|JPEG image data, JFIF standard 1.01|image/jpeg|24K`

### Post recovery tasks

This will help you more to understand the script and make your own scripts base on it. You can also put all necessary parts together into a script, modify patterns for files to search and run it. You need to create a database file with name `info-mime-size-db.txt` with information about files.

Warning:

- Remove the `echo` command in front of the `cp` and `mkdir` otherwise the script will only show what is gonna to be done without restoring anything to a destination, do a dry run. To use `echo` command is good for verify that settings for filenames and destinations looks correctly.
- Those scripts are only examples for restoration of files from folders created by photorec, be careful!

#### Head of a script

Here is a simple check to check if the `info-mime-size-db.txt` exist in the current directory to prevent possible errors with rest of the script.

    #!/bin/bash
    if [ -f info-mime-size-db.txt ]; then echo The file info-mime-size-db.txt exists continuing... ;
      else
      echo Error!! the info-mime-size-db.txt file cannot be found;exit 1;
    fi

#### Start variables

    CountAll="0"
    CountToLimit="0"
    BaseSubDirName="MyRestoredFiles"
    Destination="$HOME/NameOfBaseFolder/${BaseSubDirName}-MoreDetailsInFolderName/"
    NewDirNumber="0"
    CountToLimit="0"

#### Populate an array

Warning: Arrays become populated by reading data from a `info-mime-size-db.txt` file. Otherwise the script will not work correctly!

##### With a While loop

Here will be a short examples how to speed up of filling in arrays from a file by a pattern. To speed it up was used bash standard expressions instead of grep was used `[[ "$i" =~ "gif" ]] || [[ "$i" =~ "jpeg"]]` pattern in the `if ;then else;fi` command and awk was replaced with `${i/'|'*/}`, by avoiding of using external programs in script will make it faster. Difficulty with this script might be a more uncomfortable way of editing/creating patterns in an if command and is slower. Positive is that lines with spaces will be added to an array as a single value.

    WhileArray=0;
    while read i; do
    if [[ "$i" =~ "gif" ]]||[[ "$i" =~ "jpeg" ]];then
    ArrayOfFiles[$WhileArray]=${i/'|'*/}
    WhileArray=$((WhileArray+1));
    fi;
    done <  info-mime-size-db.txt
    echo done, the array is full

##### With grep and awk commands

This way of populating an array is many times faster as with a while command but has limitations that might cause errors. A common way of populating an array as in this example causes problems due using space between words as a separator and a file names that contains them will not be restored and errors will be shown. A $SearchFor variable is more intuitive to edit then if all patters are in the same line with grep.

    SearchFor="-e compressed -e archive";
    ArrayOfFiles=($(grep -i $SearchFor   info-mime-size-db.txt | awk -F'|' '{print $1  }'));

Without grep you have to use `if` inside of gawk command and add patterns. Suitable if file with data is a really a very big and you can chose in which part of string you want search compared to grep that uses a whole string.

    ArrayOfFiles=($( gawk -F '|' '{if ($3 ~ "image/jpeg" || $3 ~ "image/gif" || $3 ~ "image/png") print $1 }' info-mime-size-db.txt))

You can find out which of recovered files contains spaces in their names and save information about them in a file for future use.

    $ find . -type f -name "* *" >> filenames-with-spaces.txt
    $ gawk -F'|' '{if ($1 ~ " ") print $1 }' info-mime-size-db.txt >> filenames-with-spaces.txt

##### With a find command

This not required use of a `info-mime-size-db.txt` file but is limited to use only a filename patterns, e.g. extension or a part of a name and correctly use of file names that contains spaces. `ArrayOfFiles=($(find -type f -name *jpg \;))`

#### Loops for restoration

- This is a finale part of a script that manages restoration of files. When limit of files in a destination sub-directory reached then it creates and new one numbered sub-directory in the destination folder and continuing to copy files there.

    SizeOfArray=${#ArrayOfFiles[@]}
    while [  "${SizeOfArray}" != "${CountAll}" ]; do

    IfExist="${Destination}${BaseSubDirName}${NewDirNumber}"
    if [ ! -d "${IfExist}" ]; then echo mkdir -v "${IfExist}" -p;fi

    CountToLimit=$((CountToLimit+1 ))

    FileName=$(basename "${ArrayOfFiles[$CountAll]}")
        if [ $CountToLimit -gt 25 ]; then
    CountToLimit="0"
    NewDirNumber=$((NewDirNumber+1))
    fi;
    NewDestination="$IfExist"

    echo cp -fv "$PWD/${ArrayOfFiles[$CountAll]}" "${NewDestination}/${BaseSubDirName}${CountAll}-${FileName}"
    CountAll=$((CountAll+1))
    done

Note: In order to add more specific details about files in their names or names of the destination directories you will need to gather information about them with external programs, e.g. for image resolution: feh `feh -l "${ArrayOfFiles[$CountAll]}" | tail -1 | awk '{print $3"x"$4}'`, imagemagick `identify ${ArrayOfFiles[$CountAll]} | awk '{print $3}'`.

#### Enough if files are few

If it is not so many files with the same extension then it will be enough to use something like `find -name *.xcf -exec copy "{}" $HOME/Desktop \;` to avoid the overload of a destination folder you can calculate how many files are found `find -type f -name *xcf | wc -l`.
Note: The photorec utility stores up to 500 recovered files in a single folder.

### Cut generated name from original

Warning:

- This script requires the `info-mime-size-db.txt` file that was described in the begin of article.
- You can also use `find . -type f -name "*_*" >> YourSourceFile.txt` command to create an alternative source data file. In that case you will need to set YourSourceFile.txt in the `$SourceDataFile` variable.

This script will restore recovered files only which has their original names as a part of «photorec» generated name. It uses `awk` with combination of IFS internal bash variable to change array separator and make it able to populate an array even if a string contains spaces. Can be extended with patterns, is much faster than loops but is much more complicated to configure and needs some knowledge about advanced awk functionality.

Example of restoration:

    $ cp -fv ./recup_dir.940/f843558984_!_DATA_!.zip /home/user/FileWithOrigNames/files14/!_DATA_!.zip

If the destination file already exists then a Dupe123END pattern will be added in front of a filename, where 123 is number of a processed file, e.g. `‘./recup_dir.940/f843558984_!_DATA_!.zip’ -> ‘/home/user/FileWithOrigNames/files14/Dupe4813END!_DATA_!.zip’`.

Script:

    #!/bin/bash

    #SourceDataFile="YourSourceFile.txt"
    SourceDataFile="info-mime-size-db.txt"

    LimitIs="330"
    echo You have limited amount of files in a single directory to ${LimitIs}.

    echo Source file with a necessery data: $SourceDataFile
    if [ 'XX' == 'XX'"$SourceDataFile" ] ; then
     echo The '$SourceDataFile' variable is empty
     exit 1
    else
     if [ ! -f "$SourceDataFile" ]; then
      echo The "$SourceDataFile" file doesn"'"t exist'!'
      exit 2
     fi;
    fi;

    CountAll="0"
    CountToLimit="0"
    SubDir="files"
    NewFolder="0"
    IFS="|"

    Destination="$HOME/FileWithOrigNames/"
    echo  mkdir -v "${Destination}" -p

    echo Starting to fill in the array with filenames from a '«'${SourceDataFile}'»' file'!'
    ArrayOfFiles=($(awk -F'|'  '//{AA=$1;
    sub(/^.*\//,"",AA);
                BB=AA;
              if ( AA ~ "_") {
                 GUline=index(AA,"_")+1;
                 OName=substr(AA,GUline );
     print $1"|" OName"|"
    }
    else {SIName=index(AA," ");
      if (SIName){SWName=AA;
       print $1"|"SWName"|"
            }
     };
     }'  $SourceDataFile))
    echo 'Done with getting info from a file!'
    #########

    TotalItems=${#ArrayOfFiles[@]}
    while [ $TotalItems != $CountAll ] ; do

    GetOrigName="${ArrayOfFiles[$CountAll+1]}"
    FileName="${ArrayOfFiles[$CountAll]}"

    IfExist="${Destination}${SubDir}${NewFolder}"

    if [ $CountToLimit -gt $LimitIs ]; then
       CountToLimit="0"
       NewFolder=$((NewFolder+1))
     echo mkdir -v "${IfExist}" -p
    fi

    if [ ! -d "$IfExist"  ]; then mkdir -vp "${IfExist}";fi

    if [ -f "$IfExist/$GetOrigName"  ]; then
     echo cp -fv "$FileName" "${IfExist}/Dupe${CountAll}END${GetOrigName}"
      else
     echo cp -fv "$FileName" "${IfExist}/${GetOrigName}"
    fi

    CountToLimit=$((CountToLimit+1 ))
    CountAll=$((CountAll+2))
    done

Category:

- Scripts