Category Archives: Command Line (linux/bash)

Command Line (linux/bash), sysadmin, ubuntu, WSL

WSL boot shell script

Leave a comment

I’ve created this script to help me get what I need running on my WSL Ubuntu VMs

I’m calling it ~/.mrc_boot_script and I also have it saved it to: https://gist.github.com/markcerv/7ba608b69bf6edf57db456187ad8a4ff

#If we made it in here, then that's a good thing

read -r -t 15 -p "Run the only on boot commands? [y/N] " response
response=${response,,}    # tolower

if [[ "$response" =~ ^(yes|y)$ ]]
then
    echo "All of these commands need sudo, so be prepared to enter in a password"
    sleep 2

    #Need to do this to get screens running cleanly
    echo "Screen cleanup"
    sudo /etc/init.d/screen-cleanup start

    #Let's also make sure postgres is running
    echo "Fire up postgresql"
    sudo service postgresql start

    #Let's also make sure mysql is running
    echo "Fire up mysql"
    sudo service mysql start

    #Let's also make sure ssh is running
    echo "Fire up ssh"
    sudo service ssh --full-restart

    #Let's also make sure redis is running (for celery)
    echo "Fire up redis-server"
    sudo service redis-server start
else
    echo "Doing nothing"
fi
Command Line (linux/bash), Quick Fixes, Shell Related, Windows, WSL

Issues running npm & WSL2 (bad interpreter)

1 Comment

I started a new project and wanted to use node/npm inside of WSL2. However, when I tried to run npm inside of either Ubuntu 18 or 20, I got this error message:

mark@LAPTOP:~$ npm
-bash: /mnt/c/Program Files/nodejs/npm: /bin/sh^M: bad interpreter: No such file or directory

After a whole lot of searching, I came across this web page, https://stackoverflow.com/questions/63716587/in-wsl2-ubuntu-20-04-for-windows-10-nodejs-is-installed-but-npm-is-not-working which suggested updating the ~/.bashrc file to look like:

# strip out problematic Windows %PATH%
PATH=$(echo "$PATH" | sed -e 's/:\/mnt.*//g')

I added that to the VERY end of the file, and rebooted.

The next time I tried to run npm, I got this successful message instead:

mark@LAPTOP:~$ npm

Usage: npm <command>

where <command> is one of:
    access, adduser, audit, bin, bugs, c, cache, ci, cit,
    clean-install, clean-install-test, completion, config,
    create, ddp, dedupe, deprecate, dist-tag, docs, doctor,
    edit, explore, fund, get, help, help-search, hook, i, init,
    install, install-ci-test, install-test, it, link, list, ln,
    login, logout, ls, org, outdated, owner, pack, ping, prefix,
    profile, prune, publish, rb, rebuild, repo, restart, root,
    run, run-script, s, se, search, set, shrinkwrap, star,
    stars, start, stop, t, team, test, token, tst, un,
    uninstall, unpublish, unstar, up, update, v, version, view,
    whoami

npm <command> -h  quick help on <command>
npm -l            display full usage info
npm help <term>   search for help on <term>
npm help npm      involved overview

Specify configs in the ini-formatted file:
    /home/mark/.npmrc
or on the command line via: npm <command> --key value
Config info can be viewed via: npm help config

npm@6.14.4 /usr/share/npm
Command Line (linux/bash), ubuntu, WSL

Update ubuntu system clock (ntpdate)

Leave a comment

Did you notice that the system clock on your Ubuntu server is wrong? On a production server this might not happen, but on a staging server (or on a WSL instance) the date/time can dift. Here’s how to fix it:

First, ask your server what time it is (so you can have a baseline)

$ date
Thu Mar  4 12:30:02 PST 2021

If that seems wrong, you’ll want to run ntpdate…but first make sure it’s installed:

$ sudo apt install ntpdate

Next, run the command to update the time

$ sudo ntpdate time.nist.gov

# 5 Mar 10:58:20 ntpdate[11619]: step time server 132.163.97.3 offset 80783.696700 sec

Wow, I was off by over 80,000 seconds, which is 1,346 minutes or 22.4 hours!

Now that it’s been fixed, let’s check the date/time again:

$ date
Fri Mar  5 10:58:26 PST 2021

Perfect!

Command Line (linux/bash), MySQL, sysadmin

Quick mysql commands

Leave a comment 
# configure mysql 1st time
mysql -u root --skip-password
mysql>  ALTER USER 'root'@'localhost' IDENTIFIED BY 'xxx';

# create databases
mysqladmin -u root create wp_somename;

# now run mysql as root
mysql -u root mysql -p


CREATE USER 'example_user'@'localhost' IDENTIFIED WITH mysql_native_password BY 'password';

CREATE USER 'example_user'@'localhost' IDENTIFIED BY 'password';  /* in case above doesn't work */



GRANT ALL ON example_database.* TO 'example_user'@'localhost';

flush privileges;

exit;

Command Line (linux/bash), Shell Related, sysadmin

Certbot: WWW and non-WWW certificates

Leave a comment

Certbot is an awesome way to make sure your websites are secured with encryption and HTTPS.

It’s considered a best practice to have only 1 canonical version of a URL. But when you do that, you can run into issues where you get certificate mis-match errors. Here is an easy way to fix/prevent that.

sudo certbot certonly  -d originaldomain.com -d www.originaldomain.com

However, if you forgot do add in the www the first time, or if you want to create other subdomains that might answer on the same virtualhost, you’ll need to expand your original certificate.

--expand tells Certbot to update an existing certificate with a new certificate that contains all of the old domains and one or more additional new domains.

sudo certbot certonly --expand -d originaldomain.com -d www.originaldomain.com -d new.originaldomain.com -d new2.originaldomain.com -d new3.originaldomain.com --dry-run

NOTE: use the –dry-run flag for testing if it gives you the correct output without errors then re-run the same command and remove –dry-run from the end.

Apache, Command Line (linux/bash), Shell Related, sysadmin

Another instance of Certbot is already running

Leave a comment

I’ve written before about Certbot and how wonderful it is. Recently I’ve come across a situation where certbot renewals aren’t going thru:

root@host:~# certbot renew --force-renewal
Another instance of Certbot is already running.

root@host:~# ps waux | grep certbot
root     20947  0.0  0.0   4500   744 ?        Ss   16:52   0:00 /bin/sh -c /usr/local/sbin/certbot-renew.sh
root     20949  0.0  0.0   4500   736 ?        S    16:52   0:00 /bin/sh /usr/local/sbin/certbot-renew.sh
root     20953  0.3  4.4 148704 45440 ?        S    16:52   0:00 /usr/bin/python3 /usr/bin/certbot renew --post-hook touch /var/lib/letsencrypt/updated
root     21049  0.0  0.0  12940  1012 pts/4    S+   16:54   0:00 grep --color=auto certbot

root@host:~# certbot --version
certbot 0.31.0

If I check the logfile /var/log/letsencrypt/letsencrypt.log I can see that it keeps on trying and trying to do renewals.

grep "random delay" /var/log/letsencrypt/letsencrypt.log

2020-04-14 16:16:02,526:INFO:certbot.renewal:Non-interactive renewal: random delay of 5 seconds
2020-04-14 16:17:02,135:INFO:certbot.renewal:Non-interactive renewal: random delay of 409 seconds
2020-04-14 16:24:01,684:INFO:certbot.renewal:Non-interactive renewal: random delay of 290 seconds
2020-04-14 16:29:01,744:INFO:certbot.renewal:Non-interactive renewal: random delay of 190 seconds
2020-04-14 16:33:02,261:INFO:certbot.renewal:Non-interactive renewal: random delay of 283 seconds
2020-04-14 16:38:01,813:INFO:certbot.renewal:Non-interactive renewal: random delay of 350 seconds
2020-04-14 16:44:01,613:INFO:certbot.renewal:Non-interactive renewal: random delay of 117 seconds
2020-04-14 16:46:02,187:INFO:certbot.renewal:Non-interactive renewal: random delay of 309 seconds
2020-04-14 16:52:01,834:INFO:certbot.renewal:Non-interactive renewal: random delay of 453 seconds

Here’s what I do next:

  1. find any running certbot processes.
  2. kill them (with -9) one by one
  3. find and lock files
  4. remove them
root@host:~# ps waux | grep cert
root     20947  0.0  0.0   4500   744 ?        Ss   16:52   0:00 /bin/sh -c /usr/local/sbin/certbot-renew.sh
root     20949  0.0  0.0   4500   736 ?        S    16:52   0:00 /bin/sh /usr/local/sbin/certbot-renew.sh
root     20953  0.1  4.4 148704 45440 ?        S    16:52   0:00 /usr/bin/python3 /usr/bin/certbot renew --post-hook touch /var/lib/letsencrypt/updated
root     21152  0.0  0.0  12940   936 pts/4    S+   16:57   0:00 grep --color=auto cert

root@host:~# kill -9 20947
root@host:~# kill -9 20949
root@host:~# kill -9 20953
root@host:~# find / -type f -name ".certbot.lock"
/var/lib/letsencrypt/.certbot.lock
/var/log/letsencrypt/.certbot.lock
/etc/letsencrypt/.certbot.lock

root@host:~# rm /var/lib/letsencrypt/.certbot.lock  /var/log/letsencrypt/.certbot.lock /etc/letsencrypt/.certbot.lock

Next, I want to uninstall it.

root@host:~# apt list --installed | grep certbot

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

certbot/xenial,now 0.31.0-1+ubuntu16.04.1+certbot+1 all [installed]
python3-acme/xenial,now 0.31.0-2+ubuntu16.04.6+certbot+2 all [installed,automatic]
python3-asn1crypto/xenial,now 0.22.0-2+ubuntu16.04.1+certbot+1 all [installed,automatic]

Next, I want to uninstall it:

root@host:~#  apt-get remove certbot
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  python3-acme python3-certbot python3-configargparse python3-funcsigs python3-future python3-icu python3-josepy
  python3-mock python3-parsedatetime python3-pbr python3-requests-toolbelt python3-rfc3339 python3-tz
  python3-zope.component python3-zope.event python3-zope.hookable python3-zope.interface
Use 'apt autoremove' to remove them.
The following packages will be REMOVED:
  certbot
0 upgraded, 0 newly installed, 1 to remove and 9 not upgraded.
After this operation, 39.9 kB disk space will be freed.
Do you want to continue? [Y/n]

Type in Y to continue, then after that, get rid of code you no longer need

root@mhs02:~#  apt-get remove certbot
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  python3-acme python3-certbot python3-configargparse python3-funcsigs python3-future python3-icu python3-josepy
  python3-mock python3-parsedatetime python3-pbr python3-requests-toolbelt python3-rfc3339 python3-tz
  python3-zope.component python3-zope.event python3-zope.hookable python3-zope.interface
Use 'apt autoremove' to remove them.
The following packages will be REMOVED:
  certbot
0 upgraded, 0 newly installed, 1 to remove and 9 not upgraded.
After this operation, 39.9 kB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 158315 files and directories currently installed.)
Removing certbot (0.31.0-1+ubuntu16.04.1+certbot+1) ...
Processing triggers for man-db (2.7.5-1) ...
root@host:~# apt autoremove
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
  python3-acme python3-certbot python3-configargparse python3-funcsigs python3-future python3-icu python3-josepy
  python3-mock python3-parsedatetime python3-pbr python3-requests-toolbelt python3-rfc3339 python3-tz
  python3-zope.component python3-zope.event python3-zope.hookable python3-zope.interface
0 upgraded, 0 newly installed, 17 to remove and 9 not upgraded.
After this operation, 6,352 kB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 158306 files and directories currently installed.)
Removing python3-certbot (0.31.0-1+ubuntu16.04.1+certbot+1) ...
Removing python3-acme (0.31.0-2+ubuntu16.04.6+certbot+2) ...
Removing python3-configargparse (0.11.0-1+certbot~xenial+1) ...
Removing python3-mock (1.3.0-2.1ubuntu1) ...
Removing python3-funcsigs (0.4-2) ...
Removing python3-parsedatetime (2.4-3+ubuntu16.04.1+certbot+3) ...
Removing python3-future (0.15.2-4+ubuntu16.04.1+certbot+3) ...
Removing python3-icu (1.9.2-2build1) ...
Removing python3-josepy (1.1.0-2+ubuntu16.04.1+certbot+1) ...
Removing python3-pbr (1.8.0-4ubuntu1) ...
Removing python3-requests-toolbelt (0.8.0-1+ubuntu16.04.1+certbot+1) ...
Removing python3-rfc3339 (1.0-4+certbot~xenial+1) ...
Removing python3-tz (2014.10~dfsg1-0ubuntu2) ...
Removing python3-zope.component (4.3.0-1+ubuntu16.04.1+certbot+3) ...
Removing python3-zope.event (4.2.0-1) ...
Removing python3-zope.hookable (4.0.4-4+ubuntu16.04.1+certbot+1) ...
Removing python3-zope.interface (4.3.2-1+ubuntu16.04.1+certbot+1) ...

Check to see if that removed the process

root@host:~# ps waux | grep cert
root     21304  0.0  0.0   4500   748 ?        Ss   17:01   0:00 /bin/sh -c /usr/local/sbin/certbot-renew.sh
root     21306  0.0  0.0   4500   776 ?        S    17:01   0:00 /bin/sh /usr/local/sbin/certbot-renew.sh
root     21310  0.2  4.7 157436 48228 ?        S    17:01   0:00 /usr/bin/python3 /usr/bin/certbot renew --post-hook touch /var/lib/letsencrypt/updated
root     22017  0.0  0.1  12940  1092 pts/4    S+   17:08   0:00 grep --color=auto cert
root@host:~# ls /usr/local/sbin
certbot-renew.sh

root@host:~#  mv /usr/local/sbin/certbot-renew.sh   /usr/local/sbin/certbot-renew.sh.hide

Triple check nothings running

root@mhs02:~# ps waux | grep cert
root     22118  0.0  0.1  12940  1020 pts/4    S+   17:09   0:00 grep --color=auto cert

Follow the instructions at:
https://certbot.eff.org/instructions on how to install certbot for your installation.

sudo apt-get install certbot python-certbot-apache

Finally, let’s check the server uptime, check the expiration date of a site running on this server, gracefully restart apache, then check the expiration date again:

root@host:~# apachectl status | grep uptime
   Server uptime: 11 days 22 hours 45 minutes 34 seconds

root@host:~# echo | openssl s_client -servername www.example.org -connect www.example.org:443 2>/dev/null | openssl x509 -noout -dates
notBefore=Mar  3 16:27:55 2020 GMT
notAfter=Jun  1 16:27:55 2020 GMT

root@host:~# apachectl graceful
[Tue Apr 14 17:17:24.937566 2020] [so:warn] [pid 26644] AH01574: module security2_module is already loaded, skipping

root@host:~# apachectl status | egrep 'uptime|Restart'
   Restart Time: Thursday, 02-Apr-2020 18:29:10 UTC
   Server uptime: 11 days 22 hours 52 minutes 3 seconds


root@host:~# echo | openssl s_client -servername www.example.org -connect www.example.org:443 2>/dev/null | openssl x509 -noout -dates
notBefore=Apr 14 16:13:15 2020 GMT
notAfter=Jul 13 16:13:15 2020 GMT

Success! We can see that the new renewal dates are in place.

To make sure that renewals happen automatically, I like to have this crontab in place:

# Let's keep let's encrypt ssl certs up to date.
31 3,15 * * * /usr/bin/certbot renew --quiet
Command Line (linux/bash), Quick Fixes, Virtualization

Virtualbox – Your VM has become “inaccessible

Leave a comment

Today, when I tried to boot up my linux virtualbox, I got this error:

C:\home\linux>vagrant up
Bringing machine 'default' up with 'virtualbox' provider... Your VM has become "inaccessible." Unfortunately, this is a critical error with VirtualBox that Vagrant can not cleanly recover from. Please open VirtualBox and clear out your inaccessible virtual machines or find a way to fix them.

That doesn’t sound good. A bit of googling around let me to this solution., which I’ll break down into the steps that worked for me.

In my “C:\Users{user}\VirtualBox VMs{vm-id}” folder are two files

  • {vm-id}.vbox-prev
  • {vm-id}.vbox-tmp

Renaming “{vm-id}.vbox-tmp” to “{vm-id}.vbox”

Then try running “vagrant up”

BOOM. It was that easy.

Apache, Command Line (linux/bash), sysadmin

Certbot renewal script

1 Comment

By now you know that your website needs to run over HTTPS. One of the easiest and cheapest ways to do this is by using
https://certbot.eff.org/ which in turn deploys https://letsencrypt.org/ certificates.

Once you have it installed, you’ll want to install a crontab entry that will run the updates 2x per day:

23 3,15 * * * /usr/local/sbin/certbot-renew.sh
#!/bin/sh
#
# Save this file as /usr/local/sbin/certbot-renew.sh
#
UPDATE_FLAG_FILE=/var/lib/letsencrypt/updated

rm -f $UPDATE_FLAG_FILE
/usr/local/bin/certbot-auto renew --post-hook "touch $UPDATE_FLAG_FILE"

if [ $? -gt 0 ]; then
  exit $?
fi

if [ ! -f $UPDATE_FLAG_FILE ]; then
  exit 0
fi


service apache2 status

if [ $? -eq 0 ]; then
  service apache2 reload
  if [ $? -gt 0 ]; then
    >&2 echo failed to reload apache2
    exit 1
  fi
fi
Command Line (linux/bash), Command Prompt, Utilities

Creating a password protected zip file

Leave a comment

I’ve found that the easiest way to create a password protected zip file, is to use the freely available 7-zip.

C:\7z.exe a test.zip fileyouwantzipped -pPUT_PASS_PHRASE_HERE

7-Zip 18.05 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2018-04-30

Scanning the drive:
1 file, 13626 bytes (14 KiB)

Creating archive: test.zip

Add new data to archive: 1 file, 13626 bytes (14 KiB)


Files read from disk: 1
Archive size: 11376 bytes (12 KiB)
Everything is Ok

C:\

A pretty easy way to generate passwords/pass phrases is to visit: https://untroubled.org/pwgen/ppgen.cgi and change options as you see fit.