Category Archives: administration

Issue with the WordPress server.

Oh hey, look… a post. 🙂

Anyway, a couple of weeks ago, two at the time of this writing, I updated my Debian Gnu\Linux server running this site. It broke things. Instead of showing the site, the Apache server showed the text in index.php.

I was running WordPress 6.2 with PHP 8.2. Those two key points are essential because most of the documentation out there for the problem talks about Old versions of WordPress and Old versions of PHP. Neither of which fit me.

Going through sites, I found the fixes. The first was ensuring that PHP was set up in the Apache modules. The easiest way to set it up was to install Libapache2-mod-php8.2.

 

This command changed the error from the index.php to an error:
Your PHP installation appears to be missing the MySQL extension, which is required by WordPress. Please check that the mysqli PHP extension is installed and enabled.

Again, searching said to update WP and PHP (both were already at the newest), so little help. Luckily I found something pointing to  the WP support forum; someone with WP 6.2 and PHP 8.2 having the same issue. The volunteer support person, Steven Stern (sterndata) (@sterndata), said to check php-mysqlnd. The command he used, for that thread’s OP Centos box.

So one search and installation later, I’m back in business and updated to WP 6.2.2.

Now if I could just find my notes and remember everything, I was writing about CTI in January of 2022.

For those interested in root cause analysis: My last upgrade was for Testing to the newest level. It looks like some things with MySQL and php didn’t get updated right and broke the site. I just didn’t think to check if my site was working after the update.

Hello php8.1, thank you for breaking things.

This server runs Debian’s Testing release. Yes, Testing changes a lot and is not meant for long-term production servers. But I like to be a little more up-to-date on the software packages, and there are times when Debian Stable is too far out of date for what I need or want to run on this server. I do use it as a shell server too. And yes, I know bastion hosts, one “process” per server. That assumes one has a budget for multiple servers.

Anyway, I updated the server, and it pulled PHP 8.1. Previously I was running PHP7.4. Well, something interesting happened during the upgrade. I rebooted the server to get the new kernel loaded, and when everything came back up, I had the “White Page of Death” on the WordPress page. This is a self-hosted server, no CPanel, phpAdmin, or anything like that. If I have to fix a problem, it’s the command line and me.

Continue reading

Rebuilding my Chromebook’s Linux Envionment

My regular travel laptop is a 15-inch Lenovo running Gnu/Linux. A couple of years ago, I decided to get something a little smaller, lighter, and cheaper. I didn’t want to take the 15-inch laptop if I didn’t need to. I use it mostly for conference presenting and running VMs. Replacing it would be a pain.

I ended up getting an Acer Chromebook 11, the C740 model, for vacation and easier travel. I liked that model because you could replace the original storage with something larger by swapping out the SSD. I also like dit because I could install Debian to it with Crouton.

I set up the device up to Debian Buster and the xfce4-desktop. Other than not using the device enough to remember all commands to launch the chroot Linux environment, it worked well. To help remember how to launch Linux, I have the following saved

to a text file on the device.

Since it had been a bit since I used the Chromebook, I thought I would upgrade it. Heck, it was going to get an update from Google anyway. The upgrade started ok but went off the rails.

Continue reading

Dovecot + Postifx problems.

Over the weekend I updated my mail server. Turns out if you have Dovecot installed and configured with Postfix, and Dovecot fails, Postfix stops working too. When I was trying to fix Dovecot I had mail in my mailbox, I could see it if I ran the mail command on the server. But I couldn’t see the email in my desktop client. After fixing Dovecot, I couldn’t see any new email in either place.

Continue reading

more mailserver fun

I’m still working through my quarantine folders. There are about 300 emails in each folder, and there are 62 folders. The folders are named 0-9, a-z, and A-Z. I don’t know why SpamAssassin / Amavisd on Debian does it that way, but it does.

Anyway going through them one at time with zless, and then rm was a bit of a pain. So I wrote a quick little one-liner to help:

The problem is, not all of the files are in gzip format, so it didn’t display those. And going in and out of the page system for Less was an annoying flash between the pager system and the normal terminal output.

So I improved it, using zcat, because I had some issues with zgrep not supporting some grep switches, like recursive.

Now it didn’t launch the pager, so no flashing. The second thing it did was give me just the To, From, Subject, and Date fields, and I could decide to delete or not based on block of info provided. Downside was it still didn’t handle the non-gzip files.

So when I got up today, I thought why not create a shell script to do this. And I can add in the feature to release false positives that SpamAssassin put in the folders.

So I now have a Mail Administration script in my DFIR repository on GitHub, that will check if it is gzip or not. Use the right form of grep, show info, and ask what to do with the file, release or delete (or nothing if you don’t use r or d as the answer).

Still some minor issues with the script:

  1. Must be ran as root, or someone else that has access to the virusmail sub-directories. in my case that means root since the mail accounts have /bin/false set up for shells.
  2. To be more portable it has to be called from spam sub-directory. In my case spam is in /var/lib/amavis/virusmails/. Which means I have to go there, and then in to one of the 0-9, a-z, or A-Z directories first. Like so:

     
  3. I still have 300 or so emails in each folder so I’d rather work 1 folder at a time right now to clear them.

Future plans for the script:
Ask the user where their spam folder is, so the script can be called outside of those folders, and enumerate all the sub-folders.

I also have to find out if the 0-9a-zA-Z is the same for all versions of software or if that is just a Debian thing.

SPF, DKIM, DMARC, and ADSP

For a while now, I’ve been having problems with DKIM. It wasn’t working. My logs always had the same error:

And I’d look for a fix but never find anything useful.

Today I decided to go through my mail quarantine folders. In them I found several emails from a friend who is having problems with spammers using his email address. None of them are going through his mail server, they’re all spoofed. We’ve compared our SPF records and they look right. So I went and looked up why I’m seeing all these mails.

Turns out that not all mail admins have set up their servers right to look at SPF and block. That was my problem.

So I went and found a howto for my operating system to fix SPF with my Mail Transfer Agent (MTA). The document, provided by my VPS hosting provider, had how to set up SPF, how to configure my MTA to quarntine emails that fail SPF, a DKIM walk through, a ADSP howto, and a DMARC howto, all on the same page.

First things first. I fixed the SPF inbound. Now it should do the stuff it needs to. Then I figured since I still had time, I’d go after the DKIM problem.

So I backed up my existing files and followed along. AND NOTHING WORKED!. Still the same problem. Heck even the same error message.

So an

later and I started completely fresh. Nothing old, not even the old backup files.

And it still didn’t work. sysctrl status -l opendkim.service and journalctrl -xe were not much help either. Neither one gave enough information on what was wrong.

I did some searching through the logs, and found that even after changing the port to a local socet for Milter it still couldn’t work. But this time I found that it couldn’t see the files, and searching the directory that local socket should be in, it wasn’t there. After much googling I found an old bug report for Debian (my OS of choice). If the socket and pid files were missing, do this:

And suddenly everything was working. I sent test emails to test services, and they seem to be working. At least they told me that everything works.

Then I went why not and set up the ADSP and DMARC stuff in DNS.

Really just happy to get past the problem where dkim isn’t working. Now to go finish clearing out the quarantine files.

Spending a weekend re-installing my firewall appliance was not my plan

So recently while trouble shooting that mail log in problem from my phone, I started going through the web interface on my pfSense box. While in the LAN interface, and it being 4am, I was like why is block RFC1918 for the Wan (which is on every interface tab), and block Bogons, not checked.

So I did what anyone half sleep deprived would do, I checked the boxes and hit apply.  Then I couldn’t get back in to the silly thing. Console wouldn’t work. I just got a blank screen, rebooting while consoled  in would go through the post and loading of BSD but after pfSense started, I didn’t get a menu.

Hey I know, I’m a Nix person, I’ll boot from the live image, go in to recovery find and turn off that setting, sync to the hard drive and reboot.

Continue reading

Oh look HTTPS

Testing out Let’s Encrypt public beta. My thoughts so far:

90 days for the SSL certificate. Does that mean we’ll be seeing spammers setting these up to make their sites look more legit?

Mainly made for HTTPS on web servers. There is a walk through on making it work with email, using links, and some other dark Unix magic (what not everyone is running a linux mail server?). I saw a file for exchange but that’s not my cup of tea. It also brings up that whole 90 days thing again. So for now my mail server has something else.

So while the SSL Cert is good for 90 days, they tell you to update it every 60. Can’t wait to try that in 2 months. (that was sarcasm).

To be honest though, I do like the idea of Let’s Encrypt. I like that it has Debian love and script to make magic happen in the background. Less fiddling under the hood. I like that you can set up either Secure only (all HTTPS all the time) or HTTP and HTTPS. This site used to use a self signed cert for Administration, but now it’s all SSL.

Hopefully in the future we can get at least 365 day certs, more services covered, and most importantly MORE PEOPLE ENCRYPTING their web traffic.

Why is useful documentation hard to find?

I just finished reading The Linux Journal’s “Geek’s Guide to Enterprise Monitoring Success“. It was good, talking about how to leverage the monitoring to work for the IT department in an organization. This also talked about some business problems you can face, which I’ve seen first hand. I’ve been in the “metrics from another group’s monitoring tools” meeting before. Trust me, you need to be sure of yourself and what you’re doing for the company before that happens. I’ve also seen monitoring systems destroyed because the wrong people had too much access and trying to  tune the system for their needs only.

For what it was, this was a good guide. From the title though, I expected something different.

Continue reading