Scripting help

[MyWorld]

I'm busy working on an Arch package (PKGBUILD) and since I'm not a programmer it is going really slow.

I need help with the script.
To make it more elegant I want the script to check if wget already downloaded the file, and if it has to skip the download and continue with the next step. How would I go about scripting that?

This is the script so far:

#!/bin/bash
wget -P /var/cache/pacman/pkg/ http://www.bibleanalyzer.com/bibleanalyzer_4.1-0_i386.deb
deb2targz /var/cache/pacman/pkg/bibleanalyzer_4.1-0_i386.deb
tar xvzf /var/cache/pacman/pkg/bibleanalyzer_4.1-0_i386.tar.gz -C /
sed -i 's/python/python2/g' /usr/bin/bibleanalyzer

At lot left to be desired, bur I know very little when it comes to scripting...

 

[ruan567]

I'm busy working on an Arch package (PKGBUILD) and since I'm not a programmer it is going really slow.

I need help with the script.
To make it more elegant I want the script to check if wget already downloaded the file, and if it has to skip the download and continue with the next step. How would I go about scripting that?

This is the script so far:

#!/bin/bash
wget -P /var/cache/pacman/pkg/ http://www.bibleanalyzer.com/bibleanalyzer_4.1-0_i386.deb
deb2targz /var/cache/pacman/pkg/bibleanalyzer_4.1-0_i386.deb
tar xvzf /var/cache/pacman/pkg/bibleanalyzer_4.1-0_i386.tar.gz -C /
sed -i 's/python/python2/g' /usr/bin/bibleanalyzer

At lot left to be desired, bur I know very little when it comes to scripting...

if [ -f /tmp/foo.txt ]
then
echo the file exists
else
echo the file does not exist
fi

 

[MyWorld]

Thank you, so much, learning something new every millennium it seems.


Another thing, it seems I suck quite badly at even searching for the correct phrases to help me out. This is part of the script so far:

#!/bin/bash
if [ -d /home/testing/Documents/ ]
then
echo The folder exists
else
echo The folder does not exist
fi

if [ "The folder does not exist" ]
then
mkdir /home/testing/Documents
fi

if [ "The folder exists" ]
then
mkdir /home/testing/Documents/BibleAnalyzer
fi

But not everyones home directory will be the same, how to I get the correct username from the box (not super user) ?
users only show ALL the users on the box, and this far in from installing the application you would need to be su.

Basically I need this to work on any machine it is run on:
mkdir /home/testing/Documents/BibleAnalyzer

Sorry if this is a bit confusing...
the command users will show who is on the box, now I need to allow the script to create those folders in the homes of these users.

:/
Hope that came across right...

 

[fskmh]

This is from one of my post-installation scripts for a kile package I made for Slackware. I think it is very close to what you're trying to do.
You should just check that the group ID for users is also 100 on Arch. It basically just copies three config files into the users' .kde directory.

#!/bin/bash
#
# Copy kile config files to each user's home directory:
(
    echo -n "Copying config files to user directories: "
    entries=`wc -l /etc/passwd | awk '{print $1}'`
    declare -a luser[$entries]

    for i in `seq 1 $entries`; do
        username[$i]=`cat /etc/passwd | sed -n "${i}p" | awk -F: '{print $1}'`
        groupid[$i]=`cat /etc/passwd | sed -n "${i}p" | awk -F: '{print $4}'`
        if [ "${groupid[$i]}" == "100" ]; then
            if [ -d /home/${username[$i]} ]; then
                echo "${username[$i]} "
                if [ ! -d /home/${username[$i]}/.kde/share/apps/kile ]; then
                    mkdir -p /home/${username[$i]}/.kde/share/apps/kile
                fi
                cp /etc/skel/.kde/share/apps/kile/kileui.rc /home/${username[$i]}/.kde/share/apps/kile
                if [ ! -d /home/${username[$i]}/.kde/share/config ]; then
                    mkdir -p /home/${username[$i]}/.kde/share/config
                fi
                cp /etc/skel/.kde/share/config/kilerc /home/${username[$i]}/.kde/share/config/kilerc
                chown -R ${username[$i]}:users /home/${username[$i]}/.kde
            fi
        fi
    done
    echo
)

 

[MyWorld]

Daunting task you lay before a noob, but I accept the challenge!

 

[MyWorld]

GOT IT!

#!/bin/bash
#
(
    echo -n "Creating directories as required: "
    entries=`wc -l /etc/passwd | awk '{print $1}'`
    declare -a luser[$entries]

    for i in `seq 1 $entries`; do
        username[$i]=`cat /etc/passwd | sed -n "${i}p" | awk -F: '{print $1}'`
        groupid[$i]=`cat /etc/passwd | sed -n "${i}p" | awk -F: '{print $4}'`
        if [ "${groupid[$i]}" == "100" ]; then
            if [ -d /home/${username[$i]} ]; then
                echo "${username[$i]} "
                if [ ! -d /home/${username[$i]}/Documents/BibleAnalyzer ]; then
                    mkdir -p /home/${username[$i]}/Documents/BibleAnalyzer
                fi
                chown -R ${username[$i]}:users /home/${username[$i]}/Documents/BibleAnalyzer
            fi
        fi
    done
    echo
)

 

[MyWorld]

Right, this is the complete script so far, not elegant but functional.

What I can improve on is creating stops/failsafes for when an action fails to stop the whole process. Right now it just soldiers on to the next section without regard of the previous command's outcome.

Any advice? It will work as is, but to make it elegant...

#!/bin/bash

# Fetch the file
wget -P /var/cache/pacman/pkg/ http://www.bibleanalyzer.com/bibleanalyzer_4.1-0_i386.deb

# Convert the file
deb2targz /var/cache/pacman/pkg/bibleanalyzer_4.1-0_i386.deb

# Extract the file
tar xvzf /var/cache/pacman/pkg/bibleanalyzer_4.1-0_i386.tar.gz -C /

# Make it python2 compatible
sed -i 's/python/python2/g' /usr/bin/bibleanalyzer

# Create the directories required
(
    echo -n "Creating directories as required: "
    entries=`wc -l /etc/passwd | awk '{print $1}'`
    declare -a luser[$entries]

    for i in `seq 1 $entries`; do
        username[$i]=`cat /etc/passwd | sed -n "${i}p" | awk -F: '{print $1}'`
        groupid[$i]=`cat /etc/passwd | sed -n "${i}p" | awk -F: '{print $4}'`
        if [ "${groupid[$i]}" == "100" ]; then
            if [ -d /home/${username[$i]} ]; then
                echo "${username[$i]} "
                if [ ! -d /home/${username[$i]}/Documents/BibleAnalyzer ]; then
                    mkdir -p /home/${username[$i]}/Documents/BibleAnalyzer
                fi
                chown -R ${username[$i]}:users /home/${username[$i]}/Documents/BibleAnalyzer
            fi
        fi
    done
    echo
)

EDIT:
Fixed the script, I think it is ready now, only had to include "set -e" at the top.

 

[fskmh]

Looking over the script again I noticed the recursive chown and recalled that Slackware doesn't create a group by the same username when the username is created. So that chown may have to change from

chown -R ${username[$i]}:users /home/${username[$i]}/Documents/BibleAnalyzer

to

chown -R ${username[$i]}:${username[$i]} /home/${username[$i]}/Documents/BibleAnalyzer

I've only played with Arch in a VM, which I no longer have, so I don't know if this is applicable.

What I can improve on is creating stops/failsafes for when an action fails to stop the whole process. Right now it just soldiers on to the next section without regard of the previous command's outcome.

Any advice? It will work as is, but to make it elegant...

I'm not sure what the Arch packaging guidelines are like, but you could add some simple things like:

# Convert the file
deb2targz /var/cache/pacman/pkg/bibleanalyzer_4.1-0_i386.deb || exit 1

To bail out of the script if deb2targz fails for some reason, (like the file being missing).

You can also do a sanity check on a variable name to see if it's empty before trying to use it with

if [ -z $blah ]; then
    echo "Cannot parse blah - bailing."
    exit 1
fi

Be careful with this though, because -z only expects one data field, and if it encounters multiple data fields, typically from a grep that hasn't been awked it will complain.
You should probably check the Arch packaging guideline before trying to submit this to AUR or whatever.

 

[MyWorld]

I'm trying to simplify it as much as possible and I suspect I'm going a bit overboard, but it just so much fun!

Right now I'm trying to do a check with wget to see if the file has already been downloaded, and if it has to continue to the next step.

I don't know if there is any other AUR package that does the same since the package manager detects that a file has been installed and prompts you for reinstall, but where this will come in handy, and what I have missed in some AUR packages, is when the install fails at the end and you need to rerun the script it does everything over again, including downloading the file which is unnecessary IMHO (unless you specify the file to be installed beforehand).

So that is now the next step, checking to see if the file had already been downloaded and then skipping to the next command if it has.
I'm dabbling with this a bit, if I don't come right I'll post the output here again.

Thanks for all the help so far!

 

[MyWorld]

This is getting a little harder since the PKGBUILD syntax and bash script differ...

Let me first get the script going then I can adapt it to the PKGBUILD file

if [ -f /var/cache/pacman/pkg/bibleanalyzer_4.1-0_i386.deb ]
        then
                echo "File exists..."
        else
                wget -P /var/cache/pacman/pkg/ http://www.bibleanalyzer.com/bibleanalyzer_4.1-0_i386.deb
        continue
fi

Is this correct?

 

[fskmh]

What you have there is fine for debugging purposes, but for actual use I would just have the script check if the file does not exist:

if [ ! -f /var/cache/pacman/pkg/bibleanalyzer_4.1-0_i386.deb ]
        then
                wget -P /var/cache/pacman/pkg/ http://www.bibleanalyzer.com/bibleanalyzer_4.1-0_i386.deb
fi

The next step when you do the conversion to tgz will allow you the option of bailing out if the file isn't actually there.

 

[MyWorld]

Thanks for all the help!

 

[cpu.]

Firstly I know nothing when it comes to scripting, but I know a little of wget.
If you use it with -c, it will skip files already downloaded.

wget -c -P directory filename

if there's compatibility issues with -P (like with -i) then:

wget -P directory filename -c

 

[fskmh]

Firstly I know nothing when it comes to scripting, but I know a little of wget.
If you use it with -c, it will skip files already downloaded.

wget -c -P directory filename

if there's compatibility issues with -P (like with -i) then:

wget -P directory filename -c

wget -c will resume a partial download.

Perhaps you were thinking about wget -N, which will only download the source file if it is newer than the target file.

 

[cpu.]

wget -c will resume a partial download.

Perhaps you were thinking about wget -N, which will only download the source file if it is newer than the target file.

I use wget as my download "manager" of choice (384k connection).
wget -i textfile_with_urls -c

Ctrl-c to stop download, then press up to show wget -i textfile_with_urls -c, enter (to continue).

If it downloaded a file to completion - and you try it again - wget will report it as completed ("The file is already fully retrieved; nothing to do.") and move to the next URL.

 

[fskmh]

I use wget as my download "manager" of choice (384k connection).
wget -i textfile_with_urls -c

I script with wget and aria2, so I am familiar with wget -i too. I concede that wget -c does behave as you described, I use it like that myself. I was simply pointing out its documented purpose. What do you use to download an updated file with the same filename BTW?

 

[cpu.]

I script with wget and aria2, so I am familiar with wget -i too. I concede that wget -c does behave as you described, I use it like that myself. I was simply pointing out its documented purpose. What do you use to download an updated file with the same filename BTW?

I guess -N then.
The OP have a target file with version numbers in the name that will always change with updates. wget -c is perfect for that.

Disclaimer: I use wget exclusively for my downstorm downloads, no coding etc. wget -i and -c is the only hands-on experience I have with it.

 

[MyWorld]

http://www.techremedy.net/blog/17-amazing-tutorials-for-creating-bash-scripts/

 

[fskmh]

Some good stuff there. Here's the reference that got me started:
Advanced Bash-Scripting Guide - HTML
Advanced Bash-Scripting Guide - PDF
Advanced Bash-Scripting Guide - HTML tarball
I keep the html version on a local web server for quick access.

Bah, I see it's no. 2 in your list . Oh well, now we have some direct links.

 

[MyWorld]

After a lot of learning and bumping my head on the table, it is finally ready!

Here is the final product after all those late nights, and it looks nothing like we discussed in this thread!


Anyway, have a look here:
http://aur.archlinux.org/packages.php?ID=46821