Post recovery tasks
Related articles
Contents
Photorec
Creation of a file with data for arrays
In this example the xdg-mime is used to gather information about the mime types but the file --mime-type -b
and file -i -b
commands does the same output as the xdg-mime query filetype
command, with more or less details. This script will collect a lot of more additional information about the files into the info-mime-size-db.txt. Put the script in the destination directory that you used in photorec, make it executable and run it or use the sh -e start-collect-file-info.sh
command to run it.
start-collect-file-info.sh
#!/bin/bash ScriptsName=$(basename "$0") if [ 'XX' != 'XX'"$1" ]; then if [ -f "$1" ]; then echo "$1" echo "$(file "$1" -F"|" )'|'$(xdg-mime query filetype "$1")'|'$(du -h "$1" | awk '{print $1}' )" >> info-mime-size-db.txt else echo The « "$1" » is not a valid file name. fi else find -type f -exec sh -e "./$ScriptsName" "{}" \; fi
The script will build a file with pattern path to file/file name | info about the file | mime type | size, here is an example:./recup_dir.1/f872690288.jpg|JPEG image data, JFIF standard 1.01|image/jpeg|24K
Post recovery tasks
This will help you more to understand the script and make your own scripts base on it. You can also put all necessary parts together into a script, modify patterns for files to search and run it. You need to create a database file with name info-mime-size-db.txt
with information about files.
Head of a script
Here is a simple check to check if the info-mime-size-db.txt
exist in the current directory to prevent possible errors with rest of the script.
#!/bin/bash if [ -f info-mime-size-db.txt ]; then echo The file info-mime-size-db.txt exists continuing... ; else echo Error!! the info-mime-size-db.txt file cannot be found;exit 1; fi
Start variables
CountAll="0" CountToLimit="0" BaseSubDirName="MyRestoredFiles" Destination="$HOME/NameOfBaseFolder/${BaseSubDirName}-MoreDetailsInFolderName/" NewDirNumber="0" CountToLimit="0"
Populate an array
With a While loop
Here will be a short examples how to speed up of filling in arrays from a file by a pattern. To speed it up was used bash standard expressions instead of grep was used [[ "$i" =~ "gif" ]] || [[ "$i" =~ "jpeg"]]
pattern in the if ;then else;fi
command and awk was replaced with ${i/'|'*/}
, by avoiding of using external programs in script will make it faster. Difficulty with this script might be a more uncomfortable way of editing/creating patterns in an if command and is slower. Positive is that lines with spaces will be added to an array as a single value.
WhileArray=0; while read i; do if [[ "$i" =~ "gif" ]]||[[ "$i" =~ "jpeg" ]];then ArrayOfFiles[$WhileArray]=${i/'|'*/} WhileArray=$((WhileArray+1)); fi; done < info-mime-size-db.txt echo done, the array is full
With grep and awk commands
This way of populating an array is many times faster as with a while command but has limitations that might cause errors. A common way of populating an array as in this example causes problems due using space between words as a separator and a file names that contains them will not be restored and errors will be shown. A $SearchFor variable is more intuitive to edit then if all patters are in the same line with grep.
SearchFor="-e compressed -e archive"; ArrayOfFiles=($(grep -i $SearchFor info-mime-size-db.txt | awk -F'|' '{print $1 }'));
Without grep you have to use if
inside of gawk command and add patterns. Suitable if file with data is a really a very big and you can chose in which part of string you want search compared to grep that uses a whole string.
ArrayOfFiles=($( gawk -F '|' '{if ($3 ~ "image/jpeg" || $3 ~ "image/gif" || $3 ~ "image/png") print $1 }' info-mime-size-db.txt))
You can find out which of recovered files contains spaces in their names and save information about them in a file for future use.
$ find . -type f -name "* *" >> filenames-with-spaces.txt $ gawk -F'|' '{if ($1 ~ " ") print $1 }' info-mime-size-db.txt >> filenames-with-spaces.txt
With a find command
This not required use of a info-mime-size-db.txt
file but is limited to use only a filename patterns, e.g. extension or a part of a name and correctly use of file names that contains spaces.
ArrayOfFiles=($(find -type f -name *jpg \;))
Loops for restoration
- This is a finale part of a script that manages restoration of files. When limit of files in a destination sub-directory reached then it creates and new one numbered sub-directory in the destination folder and continuing to copy files there.
SizeOfArray=${#ArrayOfFiles[@]} while [ "${SizeOfArray}" != "${CountAll}" ]; do IfExist="${Destination}${BaseSubDirName}${NewDirNumber}" if [ ! -d "${IfExist}" ]; then echo mkdir -v "${IfExist}" -p;fi CountToLimit=$((CountToLimit+1 )) FileName=$(basename "${ArrayOfFiles[$CountAll]}") if [ $CountToLimit -gt 25 ]; then CountToLimit="0" NewDirNumber=$((NewDirNumber+1)) fi; NewDestination="$IfExist" echo cp -fv "$PWD/${ArrayOfFiles[$CountAll]}" "${NewDestination}/${BaseSubDirName}${CountAll}-${FileName}" CountAll=$((CountAll+1)) done
Enough if files are few
If it is not so many files with the same extension then it will be enough to use something likefind -name *.xcf -exec copy "{}" $HOME/Desktop \;
to avoid the overload of a destination folder you can calculate how many files are found find -type f -name *xcf | wc -l
.
Cut generated name from original
This script will restore recovered files only which has their original names as a part of «photorec» generated name.
It uses awk
with combination of IFS internal bash variable to change array separator and make it able to populate an array even if a string contains spaces. Can be extended with patterns, is much faster than loops but is much more complicated to configure and needs some knowledge about advanced awk functionality.
Example of restoration:
$ cp -fv ./recup_dir.940/f843558984_!_DATA_!.zip /home/user/FileWithOrigNames/files14/!_DATA_!.zip
If the destination file already exists then a Dupe123END pattern will be added in front of a filename, where 123 is number of a processed file, e.g. ‘./recup_dir.940/f843558984_!_DATA_!.zip’ -> ‘/home/user/FileWithOrigNames/files14/Dupe4813END!_DATA_!.zip’
.
Script:
#!/bin/bash #SourceDataFile="YourSourceFile.txt" SourceDataFile="info-mime-size-db.txt" LimitIs="330" echo You have limited amount of files in a single directory to ${LimitIs}. echo Source file with a necessery data: $SourceDataFile if [ 'XX' == 'XX'"$SourceDataFile" ] ; then echo The '$SourceDataFile' variable is empty exit 1 else if [ ! -f "$SourceDataFile" ]; then echo The "$SourceDataFile" file doesn"'"t exist'!' exit 2 fi; fi; CountAll="0" CountToLimit="0" SubDir="files" NewFolder="0" IFS="|" Destination="$HOME/FileWithOrigNames/" echo mkdir -v "${Destination}" -p echo Starting to fill in the array with filenames from a '«'${SourceDataFile}'»' file'!' ArrayOfFiles=($(awk -F'|' '//{AA=$1; sub(/^.*\//,"",AA); BB=AA; if ( AA ~ "_") { GUline=index(AA,"_")+1; OName=substr(AA,GUline ); print $1"|" OName"|" } else {SIName=index(AA," "); if (SIName){SWName=AA; print $1"|"SWName"|" } }; }' $SourceDataFile)) echo 'Done with getting info from a file!' ######### TotalItems=${#ArrayOfFiles[@]} while [ $TotalItems != $CountAll ] ; do GetOrigName="${ArrayOfFiles[$CountAll+1]}" FileName="${ArrayOfFiles[$CountAll]}" IfExist="${Destination}${SubDir}${NewFolder}" if [ $CountToLimit -gt $LimitIs ]; then CountToLimit="0" NewFolder=$((NewFolder+1)) echo mkdir -v "${IfExist}" -p fi if [ ! -d "$IfExist" ]; then mkdir -vp "${IfExist}";fi if [ -f "$IfExist/$GetOrigName" ]; then echo cp -fv "$FileName" "${IfExist}/Dupe${CountAll}END${GetOrigName}" else echo cp -fv "$FileName" "${IfExist}/${GetOrigName}" fi CountToLimit=$((CountToLimit+1 )) CountAll=$((CountAll+2)) done