(Re)Installing Magento 2 from shell

Sometimes, you accidentally bork your Magento 2 development database. Sometimes, you just want to reinstall as a sanity check. Luckily, you don’t have to re-extract the installer – you can simply un-install and re-install Magento 2 using the ./bin/magento CLI tool, see:

https://devdocs.magento.com/guides/v2.2/install-gde/install/cli/install-cli-install.html

However, the command needs a lot of arguments. I just dropped the following into a shell script at ./bin/reinstall.sh

[0s][/var/www/m2/bin]$ cat ./reinstall.sh
#!/bin/bash

./magento setup:install --admin-firstname Andy --admin-lastname Boyd \
--admin-email [email protected]r.com --admin-user admin \
--admin-password Password1! --base-url http://m2.local --backend-frontname m2_admin \
--db-host localhost --db-name m2 --db-user root --db-password Password1!

Magento 2 and PHP Unit – ‘no tests executed’ issue

When starting on unit tests for a Magento 2 module, It seemed that phpunit refused to find any of my test methods. After some experimentation, I found a fix (well, maybe a workaround)

– copy (magento root)/dev/tests/phpunit.xml.dist to ./phpunit.xml, and then pass that as the config file (this was pretty obvious)
– you have to call Magento’s phpunit binary, not your system phpunit binary (this was way less obvious)

So where you could normally do something like

cd /path/to/magento2/app/code/YourCompany/YourModule/Tests/Unit ;
phpunit . ;

You instead need to do

cd /path/to/magento2/app/code/YourCompany/YourModule/Tests/Unit ;
./vendor/phpunit/phpunit/phpunit -c /path/to/magento2/dev/test/unit/phpunit.xml .

The problem with just doing ‘phpunit’ is it calls /usr/bin/phpunit, or whatever your local system phpunit is.

For convenience, you can replace the long file paths above with a couple symlinks and save yourself a LOT of typing. Become root on your system and do

cd /usr/bin ;
mv ./phpunit ./phpunit-system ;
ln -s /path/to/magento2/vendor/phpunit/phpunit/phpunit ./ ;
cd /path/to/magento2/app/code/YourCompany/YourModule/Tests/Unit ;
ln -s /path/to/magento2/dev/tests/unit/phpunit.xml ./ ;

You replaced the usual phpunit system binary with a symlink to Magento’s, and added a symlink to the phpunit config file in your modules unit tests directory. So now to run your module’s unit tests, just do

cd /path/to/magento2/app/code/YourCompany/YourModule/Tests/Unit ;
phpunit -c ./phpunit .

πŸ˜€

Speed up VmWare virtual machine by changing location of nvram (virtual RAM) file

I’ve got multiple VMs on a HDD RAID array. I/O is ok, but it’s platter style drives in RAID-1 – speed is constrained by the limitations of the drive design.

I also have a small SSD RAID array of only 80G that houses the base Windows O/S, it idea being that the base O/S should run as quick as possible. Obviously, with a Win10 install, I’ve only got a few GB to spare on that partition, so there’s no way I could move the VMs over.

Lightbulb: I could still increase VM performance many times over if I could have VmWare use the SSD array for *just* the virtual memory file.

Luckily, the VM config file has just an option for this:

workingdir = "c:\VmWareWorkingDir"

Now the VM disk images still live on the HDD array, but the virtual memory file now lives on the SSD array – with much faster I/O πŸ˜€

Prevent Magento 1 admins from sharing credentials with Admin Single Session

Just released a free Magento extension designed to prevent admin users from sharing credentials (which wrecks accountability when things go wrong). It prevents an admin account from having more than one active session at a time. If a second user logs in using the same username and password, the first user gets redirected to the login screen with a nice message explaining they’ve been kicked because another user logged in with the same credentials.

It’s just a beta release, not thoroughly tested across different Mage versions (developed on CE 1.9.2.2), but the extension is simple enough it should work fine in most cases. Obviously, as with everything, test before pushing to production.

https://github.com/siliconrockstar/magento-admin-single-session

Thanks go to Jared (http://molotovbliss.com/) for adding modman support πŸ™‚

UPDATE: awesomely enough, this feature is baked-in to Magento 2 πŸ™‚

Using the stress command to generate CPU load

I needed a reliable way to test a script I wrote to monitor server load. Luckily I found it in bash’s stressΒ command.

On most versions of Linux, you can install it with

yum install stress

or

apt-get install stress

depending on your distro.

I was on Centos 7.2, which of course doesn’t have the package in a repo (including EPEL), so I downloaded it from here

ftp://fr2.rpmfind.net/linux/dag/redhat/el7/en/x86_64/dag/RPMS/stress-1.0.2-1.el7.rf.x86_64.rpm

and did

yum localinstall stress-1.0.2-1.el7.rf.x86_64.rpm

Usage is straightforward, all I needed to do was generate 90 seconds of greater than 70% CPU usage on a one CPU cloud virt, so I did

stress --cpu 2 --timeout 90

Shell script to monitor server load

Don’t feel like shelling out $$$ for NewRelic or Blackfire.io? Got a micro instance hosting a blog that nobody reads that doesn’t have the memory to support enterprise monitoring anyway (*cough* this blog *cough*)?

Here’s a shell script that will email you when server load gets above whatever threshold you specify. Would be pretty easy to adapt to monitor memory using the ‘free‘ command as well. Just schedule it using ‘crontab -e‘, like ‘*/5 * * * * /path/to/script.sh‘ for every 5 minutes, and you’re set.

Server load monitoring on a budget!

#!/bin/bash
# requires bc library - yum install bc

# config
# alert threshold, as decimal
ALERT=.7; # 70% CPU utilization
# admin email
EMAIL='[email protected]';

# add /usr/bin to path so cron works
export PATH=$PATH:/usr/bin;

# get number of processors
NPROC=`nproc`;
# get first utilization metric
UTIL=`cat /proc/loadavg | cat -d ' ' -f 1`;

# divide util by number of processors, accounting for 0.00 util
RESULT=`bc <<< "scale = 2; $UTIL / $NPROC"`; 

# email alert if util is greater than alert threshold 
if [[ `bc <<< "$RESULT > $ALERT"` -eq 1 ]]
then
  # calculate a percentage
  PERC=`bc <<< "scale = 2; $RESULT * 100"`;
  echo "CPU utilization is above threshold at $PERC %";
  # add top output to email
  TOPOUTPUT=`top -n 1 -b`;
  `/usr/bin/mailx -s "Utilization high on $HOSTNAME" -r "$EMAIL" "$EMAIL" <<< "CPU on $HOSTNAME is at $PERC % 

$TOPOUTPUT"`;
# else
  # echo "Utilization is only $RESULT";
fi

You can test it by generating CPU load with the stress utility if you like.

Google Analytics – take daily with a grain of salt

Haven’t messed with Google Analytics in literally years, but since I’m getting a new site up I logged in. I noticed these spikes on all of my accounts for different sites, even sites that do not exist anymore:

WtfGoogleAnalytics
Most of the traffic was from Great Britain. I’m sure there’s an explanation… that I don’t have the time or motivation to uncover.

My point is – approach your analytics data with a touch of healthy skepticism.

Real Life Security Fails – How a Lost Password Sent Thousands of Magento Credit Cards to Eastern Europe

All of the server security on the planet is useless if you don’t keep track of your passwords.

One of the most effective yet hardest to detect attacks against Magento I’ve seen to date requires nothing but a medium-level admin login and the ability to copy/paste an 8th grade level piece of code. Best (worst?) of all, you won’t even know if you’re infected, because it causes no obvious changes in the site, neither creates nor edits any files, slides under every website malware checker I know of, and doesn’t even show anything shady in webserver access logs.

I have seen this exploit quietly shuttle customer identity and credit card data to attack servers for literally *months* before anyone noticed.

Also, I don’t mean to pick on Eastern Europe, but if we’re being honest, most good malware comes out of Russia/China/Eastern Europe (Ukrainian Magento developers are some of the best). Blame a high cultural importance placed on STEM education and a fluctuating economy. The times I’ve encountered this hack in the wild, it’s usually been shuttling data to that general area.

The hack goes about like this:

1. Attacker gets a hold of an admin login, maybe by brute-force password guessing, but more likely by getting a hold of an email or some other electronic correspondence, probably from a trusted or formerly-trusted 3rd party like an overseas developer.
2. Attacker logs into admin.
3. Attacker goes to System > Configuration > Design > Footer

MagentoAddMiscFooterJavascript

4. Attacker pastes in a piece of javascript that gets loaded in the footer of every page, that basically simply says

if this is the checkout page
when the submit button is pressed
grab the customer’s data,
create a URL like http://attacker-domain.com/collector.php?firstname=John&lastname=Doe&CcNumber=1234…
and send a request to that URL.

The most devious part of this, besides the fact that it takes minimal technical skills, is that the script doesn’t actually load anything into the page, so malware scanners won’t notice it. All the attacker has to do is write a script on their own server that grabs the data out of the URL parameters and they’ve got all the customer’s identity information and their credit card number.

Counter Measures

1. AFAIK, the only 100% effective countermeasure against this hack is to do a checkout with your browser running through a proxy, record all network traffic, and then verify the identity of all the servers involved.

2. A better-than-nothing test is to request the page via script, preferably by driving a headless browser like PhantomJS, use some regex magic to parse out any URLs, and then verify the server location via IP-to-country API.

3. Rotate passwords and keep track of who is logging into your Magento.

  • I wrote some shell scripts that perform the second option, but unfortunately I don’t own the code so I can’t release them πŸ™
  • I’m working on a Selenium/NodeJS system that performs the first (superior) check, and I promise I’ll either open source the components or provide a free version πŸ™‚
  • I just finished an early-but-functional version of a free extension that logs all of your admin area logins -https://github.com/siliconrockstar/magento-admin-login-logFuture versions may support IP geolocation. I’ve also got plans in the works for other extensions that add enterprise-level admin user security features like password rotation enforcement, adjustable password difficulty enforcement, etc.

Nginx and php-fpm – ‘File not found’

Ran into this issue briefly when moving a blog over to a Centos 7.2 cloud box.

File not found.

Nginx was running as user ‘nginx’, and that user had full ownership of all web files. Logically, everything should have run fine with 700 permissions, but it wasn’t.

cat /etc/passwd

showed a user named ‘apache’. Why would that user exist on a system that had never had Apache installed?

Ahhhhh… php-fpm. The default php-fpm user is ‘apache’ – I guess just assuming it will be installed alongside apache is easier than trying to figure out how to have the webserver and php-fpm run in the same group or something.

Edited /etc/php-fpm.d/www.conf to set php-fpm’s user and group to ‘nginx’, then did

chown -R nginx.nginx /var/www/webroot
chmod -R 500 /var/www/webroot
chmod -R 700 /var/www/webroot/wp-content/uploads

Now the blog runs fine and permissions are as tight as possible without breaking things (so far lol).

Disabling PHP on a per-directory basis

Any PHP web application that allows uploads, by necessity, is going to need to vary permissions on a per-directory basis (unless you want to just save a hacker some work and make the directories that contain code files writeable by the webserver). Magento needs to be able to write to ./var, ./media, and also ./app/etc if you want to run the installer.

The ./media/ directory is an exceptionally temping gap in the application’s armor, because files there are also web-accessible out of necessity. That means if an attacker can manage to write a code file there, they can then execute it at will by simply requesting the file in a browser or sending an http request another way.

One of the easiest and most effective countermeasures against this method of attack is to simply block php execution in directories that should not contain php files. While it’s tempting to use an .htaccess file (.htaccess files in themselves kind of suck for both security and performance but that’s a diatribe for another day), there is a high risk of an attacker just overwriting it with a new, looser .htaccess file.

Instead, use the following in your Apache virtual host file (test these first, they’re tweaked snippets I found online but they look about right)

<Directory /media>
    <FilesMatch "(?i)\.(php|phtml)$">
        Order Deny,Allow
        Deny from All
    </FilesMatch>
</Directory>

… or your nginx config file

location ~* /media/.*\.(php|phtml)$ {
    return 404;
}

That should block any files with .php or .phtml file extensions in the directory from being parsed. You might want to do the same for ./var/, depending on the installation – the preferable action is to hide ./var/ from the web completely, but some installs with import/export extensions and other 3rd party code require allowing some access.