tech: the [ultimate] home server on the cheap

After having a couple Raspberry Pi’s doing miscellaneous things, and working my way up to a Plex server where I needed real storage and video transcoding crunching power, it was time to do something real. But I didn’t want to spend a lot of money. Here’s what I did.

Hardware

I wanted a decent machine that had more horsepower than a Pi, a 64-bit CPU, space for multiple mass storage 3.5″ internal drives, was relatively quiet, and relatively inexpensive. A used PC tower would fit this bill nicely. After some hunting around on Newegg for refurbished systems, I settled on a Dell Optiplex 790 mini tower. For $168 I got a machine with space for 4 hard drives (2×3.5 + 2×5.25), a 2nd-gen Core i5 at 3.10GHz, 8GB RAM, 4 internal SATA ports, and gigabit ethernet. It did come with a single 500GB drive with a fresh copy of Windows 10 Pro preinstalled, but I replaced that with a used 2TB disk I had laying around the house upon which I installed Ubuntu 20.04 LTS to make this a headless server (no monitor/keyboard attached, just sshd).

For mass data storage I bought two Seagate Ironwolf 8TB NAS drives at $200 each. The NAS drives are designed to be on consistently and accumulate a lot of spin time, as opposed to a generic PC disk that are expected to be powered off when not in use.

Then I did some re-arranging of the hard drives.

First, take the OS disk that is in the bracket at the bottom of the case, and mount it in the 5.25 bay right below the DVD drive using a 5.25-to-3.25 adapter like this one. Now the two drive brackets at the bottom of the case are empty, which I’ll get to in a minute.

Second, pull out the DVD drive that came with that refurbished PC. Replace it with a bracket for $20 that will allow a random 3.5″ internal drive to slide in to a hot-swap SATA slot. That way I can use other various internal-style drives on-demand without having to open the case. Useful for wiping that stack of hard disks from previous PCs that I’ve been meaning to do. Ubuntu includes support for NTFS filesystems, so a simple mount command can make Windows data available here to migrate in to my mass storage disks. To bring the inserted drive online without a reboot you can use the command “/usr/bin/rescan-scsi-bus.sh” as root from the sg3-utils package via “apt install”. But to do so, I believe this bracket needs to have been powered on at boot time.

Third, now that the two hard disk brackets are open at the bottom of the case, that is where I want my two mass storage drives to go. I plan to run them in a mirrored configuration, to protect against drive failure. So I aim to put the two Ironwolf drives there. But Dell uses a blue plastic caddy tray to hold the disks here instead of old-school screws, and only one caddy is present. No worries, I can get another one just like it from Newegg marketplace for $12.

For the SATA cables, I attach the OS disk to SATA port 1 on the motherboard, the two mass storage Ironwolf drives to SATA port 2 and 3 on the motherboard, and the hot-swap bracket to SATA port 4. That way when I use the hot-swap drive and power it on and off using the built-in switch on the front of it, it will get assigned the /dev/sdd device name at the end of the list, and not impact the sda assignment for the OS disk or the sdb/sdc assignment for the mass storage disks.

After I unboxed the refurbished Dell upon arrival, it wouldn’t boot up to the POST / BIOS menu. I was worried I was going to have to return it as being DOA. As a bit of a wild guess I started re-seating the internal components, and after re-seating the RAM DIMMs, it finally booted. Easy enough for a $168 server.

Operating System

It’s got to be Linux. It’s what I’m familiar with, is highly configurable, and has lots of network application software. For the sake of ease, I chose Ubuntu 64-bit server 20.04 LTS so that I can have some stability without having to do major upgrades. Running “apt update + apt upgrade” is pretty darn easy. Download the ISO from the web site (“Manual server installation”), write it to a USB thumb drive with Etcher, boot the PC from the thumb drive via the BIOS menu, install to the OS hard disk, and in short order I’ve got Ubuntu up and running. Create a login id and don’t forget the password.

Now for the mass storage drives. I’d like them to be mirrored, so in case one drive has a catastrophic failure, my data is still intact. I’ve heard lots of good things about ZFS, so let’s give that a try. It’s powerful but can be simple. And it helps that Ubuntu has support for ZFS and instructions for setting it up. I create a mirrored pool with sdb and sdc, use the “lsblk” command to verify the device names, looking for the ~8TB devices:

$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop1 7:1 0 55.4M 1 loop /snap/core18/1944
loop2 7:2 0 55.4M 1 loop /snap/core18/1932
loop3 7:3 0 69.8M 1 loop /snap/lxd/19032
loop4 7:4 0 31.1M 1 loop /snap/snapd/10707
loop5 7:5 0 31.1M 1 loop /snap/snapd/10492
loop6 7:6 0 69.9M 1 loop /snap/lxd/19188
sda 8:0 0 1.8T 0 disk
├─sda1 8:1 0 500M 0 part
├─sda2 8:2 0 465.3G 0 part
└─sda3 8:3 0 1.4T 0 part /
sdb 8:16 0 7.3T 0 disk
├─sdb1 8:17 0 7.3T 0 part
└─sdb9 8:25 0 8M 0 part
sdc 8:32 0 7.3T 0 disk
├─sdc1 8:33 0 7.3T 0 part
└─sdc9 8:41 0 8M 0 part
$ zpool status
pool: pool0
state: ONLINE
scan: scrub repaired 0B in 0 days 02:05:31 with 0 errors on Sun Jan 10 02:29:33 2021
config:
NAME        STATE     READ WRITE CKSUM
pool0       ONLINE       0     0     0
  mirror-0  ONLINE       0     0     0
    sdb     ONLINE       0     0     0
    sdc     ONLINE       0     0     0
errors: No known data errors
$

Now I create the ZFS datasets, all under the /data mount point. Here is how it looks when done:

$ zfs list
NAME USED AVAIL REFER MOUNTPOINT
pool0 1.73T 5.31T 104K /data
pool0/logs 2.08M 5.31T 2.08M /data/logs
pool0/plex 1.73T 5.31T 1.73T /data/plex
pool0/preserve 376M 5.31T 376M /data/preserve
pool0/smbshare 36.7M 5.31T 36.7M /data/smbshare
marcelk@filesvr:~$

Applications

Now what server applications to install and use? This is where it gets to be fun.

Plex

First, the main driver for this server is for Plex. I started off playing with Plex server on the family Windows PC for music with their free license to try it out. But I waited until Plex had a sale on their Lifetime Plex Pass, had a good experience and am ready to move up. So, start with installing Plex from the official repository via apt, and I can automatically get updates too, which seem to come out every couple of weeks. Then I shutdown the trial one on the Windows PC, and move all my content over into the usual Plex naming scheme for directories and files and for extras.

Why Plex? For starters, for music management. Instead of using a subscription service where I can pick any song and stream it, I have pretty specific tastes. I like to buy MP3s online and stick with those. I buy only a couple new songs a month that I like, which is much less money than a subscription service. And I’ve ripped my old collection of audio CDs. My main use case is that I don’t stream music to my phone to play, but I’d rather download the whole library and play locally, so that I don’t incur cell data usage when in my car or otherwise out-and-about. (I get by on 1GB of cell data a month, pre-pandemic.) Rip your CDs into the lossless FLAC format, and Plex will automatically compress it down to the desired bit rate in an MP3 file for local synchronization to your portable device. And with Google discontinuing Google Play Music, where I had uploaded my MP3 library for syncing back to my phone, and liked it, I wanted to exit cloud-based providers and just manage it myself. Plex offers a pretty good system for managing your music on a local server, and syncing to devices. I hadn’t really thought much about ripping movies to host in Plex, but after a trial that looks to work pretty good too. So might as well get a Blu-ray drive for the family PC where I can run MakeMKV in Windows to rip DVDs and Blu-ray discs and host those in Plex too. A ripped DVD is about 5GB, and a ripped Blu-ray is about 30GB. So in 1TB I can fit about 166 DVDs or 33 Blu-rays. I can run the Plex app on my Roku to watch these movies on my big screen TV in HD, or on any other local device using the Plex app, or on a PC using the browser interface. Because my TV’s old-school physical Blu-ray disc player does such a good job automatically up-sampling a DVD (480p) to HD (1080p), it’s still best to use the DVD discs on that big screen. But on a phone or tablet, the lack of upsampling is not very noticeable.

And I should mention since I don’t have cable TV, I’d like to be able to record over-the-air TV with an antenna and have it appear in Plex as a recorded TV show. That is pretty easy to do with a device like the HDHomeRun Connect Duo for $100. In retrospect, since I have stations in my area broadcasting in ATSC 3.0, I should have upgraded to the HDHomeRun Connect 4K for another $100 more. Once I have the Plex server up and running, it is really easy to get the Plex server to send commands to the Connect Duo to have it record shows based on the schedule I set up in Plex, copy the show from the Connect Duo stream and store it on Plex, and play it back on-demand in Plex. It’s a DVR!

And as I digitize some old family photo albums with a flatbed scanner, I can host those in Plex also. (My phone camera photos have been going into Google Photos, but maybe I’ll export them down to my Plex server too.) So I’m using Plex for music, playlists, movies and other shows from discs, recorded TV shows, live TV, and some photos.

Plex does have the capability to enable remote access to your server, to allow your phone or other devices to reach your Plex server on your home network from the outside. I did not enable that, because I don’t want to have any possibility of a connection being initiated from the outside to be able to reach inside my home network. That is what my firewall is for (accept established+related, otherwise drop). Yes, Plex will say that their solution is good, but all software has bugs. Here’s a recent one. You may be willing to take the risk to get the benefit, but I’m not, and instead will use Plex’s sync capability so I can play music and pre-selected video content while away from home, instead of making it reachable from the Internet for streaming.

httpd: nginx

Next, of course I’ll need a general purpose web server for whatever documents I want to have available on my home network. Nothing fancy, perhaps a home page with pointers to other places, such as the webgui admin consoles for my devices. For that I’ll choose nginx. And I’ll use the default index.html in /var/www/html for now, and let it appear on port 80. I don’t really care about SSL inside my home network, so I won’t bother to set up port 443 service and certificates. Since the Plex server runs its web interface on port 32400, might as well as add that as a first link in the index.html file on my new home page.

I do have a collection of product manual PDFs that I’ve collected, so I don’t need to keep the paper copies around. Let’s upload those to the /data/preserve/Manuals directory, and add a location stanza to the /etc/nginx/sites-available/default

    location /manuals/ {
            alias /data/preserve/Manuals/;
            autoindex on;
            autoindex_exact_size off;
    }

WiFi AP Manager

I use WiFi access points in Ubiquity’s Unifi product line. They require a manager application (controller) for configuration, reports, and software updates. It is not small. I had been running that controller on a Pi 3, would be nice to consolidate it on this server. I install that application from their repository so I continue to easily get updates via apt. They seem to update every couple of months. I export the controller’s config to a file, copy that file from the Pi 3 to my new server, import it into the new controller, and shutdown the old controller on the Pi 3. This webgui runs over https on port 8443, might as well add that as another link in the index.html on my nginx home page.

mDNS / Avahi

This is something you need to install, but it will “configure” itself via the applications that use it. Avahi provides an implementation of multicast DNS (mDNS) and DNS Service Discovery (DNS-SD), perhaps better known as Bonjour or Rendezvous or zeroconf. The avahi-daemon is probably already installed, and there really isn’t any config you need to do. Want to use the avahi client to see what services are on your LAN? (Use ctl-C to stop after a while, it will keep listening forever)

$ avahi-browse --all --no-db-lookup
$ avahi-browse --all --resolve
  • _googlecast._tcp: devices that can you can Google-cast to
  • _ipp._tcp: CUPS printer (Internet Printing Protocol)
  • _afpovertcp._tcp: Apple File Sharing / AFP server
  • _smb._tcp: Samba server or Windows file sharing

Windows File Sharing: Samba

Yes, there is a family / gaming PC running Windows 10, so some kind of Windows file sharing (SMB) is needed. And honestly, MacOS seems to play decently with SMB, so it is a good lowest-common-denominator. And heck, the Linux SMB client works too. Follow these instructions to get Samba server set up. I use pretty much the default config, and edit /etc/samba/smb.conf to add the following shares. The first is just a general scratch space that is readable and writable by everyone (easier than a USB stick), and the other two are simply read-only raw access to the ripped movies (mkv) and music (mp3) files.

[share]
comment = Shareable Space
path = /data/smbshare
browseable = yes
read only = no
guest ok = yes
create mask = 0666
directory mask = 0777
[music]
comment = Music MP3s
path = /data/plex/Media/Music
browseable = yes
read only = yes
guest ok = yes
[movies]
comment = Movie Discs
path = /data/plex/Media/Movies
browseable = yes
read only = yes
guest ok = yes

Apple file sharing / AFP / netatalk

My spouse is an Apple fan: a big iMac desktop, iPhone, a couple iPads, AppleTV, you get the idea. Would be nice to provide Apple file shares for her, especially when backed by the same common storage as Samba for cross-platform sharing. I could set my server up to also act as a Time Machine backup server for her too, but she has enough local USB drives to handle that. Netatalk will give me the Apple File Protocol (AFP) server on Linux.

$ sudo apt install netatalk

Then edit /etc/netatalk/afp.conf. Here is what mine looks like, I didn’t bother to enable Time Machine. And note that the “mimic model” value will make my server appear on my wife’s client as an icon of a rack-mounted Mac, using the name in the “zeroconf name” setting. (Hmm, this feels similar to the Samba configuration.)

[Global]
; Global server settings
uam list = uams_guest.so, uams_dhx.so, uams_dhx2.so
guest account = nobody
mimic model = RackMac
zeroconf name = file server
; log level = default:debug
[Share]
path = /data/smbshare
case sensitive = yes
read only = no
file perm = 0666
directory perm = 777
[Plex Music files]
path = /data/plex/Media/Music
case sensitive = yes
read only = yes
[Plex Movie files]
path = /data/plex/Media/Movies
case sensitive = yes
read only = yes

Logging

And since we have plenty of storage here, might as well set up a logging server so all my logs are centrally located. So rsyslogd will take care of that for me.

UPS / Battery Backup

I have a UPS (battery backup) connected to my broadband modem, firewall, and one WiFi AP. So in the event of a power loss, I’ll still have internet+WiFi for a bit, at least for about 6 hours until the battery runs out. Since this server is physically located right there, it would be nice to integrate my server to the UPS, so I can automatically gracefully shutdown my server when the power is out for more than 60 seconds. I have an APC BackUps 1500, and attached the cable that came with it to a USB port on my server. Here is the config I use for that:

# apt install apcupsd
# vi /etc/apcupsd/apcupsd.conf
  UPSNAME 1500
  UPSCABLE usb
  UPSTYPE usb
  DEVICE
  TIMEOUT 60
    
# apcaccess status

Printer Server

Because my family laser printer is located elsewhere, out of reach for a USB cable, I have a Raspberry Pi with Ubuntu acting as a print server for all my machines. CUPS is the way to do it. I can print from Windows, MacOS, Linux, iOS, and Android. I still find it pretty cool to print to paper from my phone, I’m old school.

DHCP with static assignments

This part may be a case of me being a neat freak. I currently have 41 devices on my home network, all of them set up for getting an IP address dynamically instead of statically. So if something goes wonky and I see an IP address behaving weird, I’d like to know what device it maps to. For the router I use (a Mikrotik hEX, which is awesomely configurable and performant for my 1GB fiber ISP) I can show the list of active DHCP leases, and convert each one to a static IP address over DHCP based on its MAC address. And I can put each one within a chosen range. The current ranges I have are for local network infrastructure (i.e., router, smart switches, WiFi access points), server / PCs / printers (including Chromebooks and laptops), media devices (Chromecasts, Rokus, etc), and smartphones / tablets. Every once in a while I’ll look to make sure the list of dynamically assigned IP addresses outside my static ranges is empty.

Removing Ubuntu cloud-init

With Ubuntu installed to bare metal instead of as a virtual image, I don’t need cloud-init running, and the messages get a little bit annoying. So remove it.

tech: software development principles

After working on a project for a while with Agile and analyzing why some things went wrong, I came up with the following guidelines for our team. I printed them out and posted it on my cube wall so I would see it multiple times daily. I see these as supplements to or applications of the Agile principles. I don’t think these are unique to our project, so I’ll share them with anyone who can get value from them.

Here is my list:

Do the right thing, even when it takes more time than the easy thing. In the best case, not doing the right thing creates debt. It is better to do a few things well than a lot of things poorly.

Obey the model. Perfectly. Models exist to provide rules of operation. Where rules exist, assumptions get made. When the model is deviated from, even in small amounts, assumptions get broken, unexpected behavior occurs, and complexity ensues. And stuff breaks.

Always keep it simple. Complexity is our #1 enemy. We must be diligently fighting this enemy wherever it silently creeps in. It is much easier to achieve the *-abilities (maintainability, usability, etc.) when the thing is simple. It’s corollary: pre-optimization is the root of all evil. Keep it simple, and then optimize only what the data tells you is actually needed – it’s likely less than you think.

Prioritize. And revisit the priorities. Start with the understanding that you can’t complete everything. Work on the most important things first. Front-load the risky items so you have time to deal with unexpected bad outcomes. Priorities will help you decide what to leave undone, as you will have to leave some things undone. Revisit the priorities periodically to check if they need to be adjusted, especially due to new information.

Work on the right thing at the right time. This is a big one. If we start with the assumption that there will always be more work to do than time allows, we can’t do it all. So pick the right thing that is most important at that moment. And the right thing will vary based on a lot of factors. It varies over time, usually a lot.

The goals should be clear. If they aren’t, ask. I want my team to understand what the high-level goals are, and how the work they do on a daily or hourly basis helps the team to achieve those high-level goals, and why that work is important to the team. If you can’t answer that positively, then either you don’t understand the goals, you don’t understand the task, or you’re working on the wrong thing. It’s totally cool to raise your hand and say “Help me understand the goals.” It shows that you care and that you are a team player. The only stupid question is the one that is not asked.

Make the best customer experience. Don’t trade “what is best for the customer” with “what is easy for us”. Pleasing the customer over the long term is your revenue source.

Think hard. And ask for advice. Challenge yourself to come up with a better plan, even after you’ve come up with a good plan. Benefit from the knowledge and diversity of those around you. There is value in personal interactions. Don’t be a heads-down soloist.

Any task which is more than trivially repeated, especially if accuracy is important, and can be reasonably automated, must be automated. The only way to scale without a huge increase in human resource cost and human errors is to automate. This also frees you up from the mundane to focus on high-value items. In the long run, the investment will not only be worth it, it will be necessary.

Break up the large stories into digestible-size pieces. If the size of the story is too big, you won’t get it done, and it will just keep rolling over iterations. The way to eat an elephant is one bite at a time, properly swallowing between each bite.

Exercise due diligence, but don’t spin your wheels. Before you go to someone to ask them a question, do some homework. But don’t spend so much time doing homework research that you don’t get the job done in a reasonable time. If you get stuck, back up and take a look at it, and if you still can’t figure it out, then go ask for help. Asking for help is totally OK. You would gladly provide help to someone if asked, right? This is a team, not an set of isolated individuals. Instead of brute forcing something, try to work smarter. Be efficient.

Too much communication is better than too little. As developers, we may not be the best communicators. But communication is essential to a team functioning properly, so we can all do the right thing at the right time. The team’s goals is bigger than what any one person can do. When in doubt if you should let someone know about something, always let them know.

Context is everything. If you understand the context, you will naturally choose the right thing. If you don’t understand the context, you’ll be making a decision blindly. In these complex environments, making good decisions is critical. Understand the context.

When it can be done reasonably, write tests to go with your code. I’m not suggesting about going crazy with unit tests, in my experience that’s not what usually breaks. With solid developers, it’s the integration function that cuts across components is the more frequent issue. But those are also generally hard to automate testing for. But if you can automate integration testing, then you can run it frequently and inexpensively, which will yield great payoffs in bug finding and QA cost savings. Scale!

Don’t let “perfect” get in the way of “done”. There absolutely will be times where your code doesn’t need to be perfect. For example, perhaps the primary business goal is speed-to-market, and the product is expected to remain pliable so customer feedback can guide its future direction, and it is understood to not be mission-critical for the customer. So don’t invest time in becoming perfect because the product is expected to change, and you’ll end up changing that code anyway. “Doing the right thing at the right time” can mean that you are building things to be good enough, not perfect. Rules, including the ones here, should always be interpreted with an “as reasonable” clause.

Test your code before you merge it. Have proven confidence that it works instead of expecting QA to catch basic problems. We currently have a dedicated QA team. But don’t get lazy and expect them to catch issues in your code. The QA team finding a problem in your code should be an exception and not the rule. The QA team is generally better at finding edge cases and interesting integration scenarios. But you as the developer should do enough testing of your own code in your development environment before you git commit + push that you have confidence in your code because you saw it actually do the right thing, not because you are arrogant.

Understand the “iron triangle”. For a product that is considered mature and customers have high reliance on working properly (i.e., bank transfer), quality needs to be there. In the iron triangle, the 3 vertexes are resources (including time and people), content, and quality. If  you move one vertex, the other two will also need to change. If you choose to increase the quality, it may take more resources and/or less content. In my current position, we generally have flexibility for time, but generally not flexibility for quality. If you have a product where time-to-market is important and shortened, then quality and/or content will also be lessened.

On average, go at a sustainable pace. There will be peaks and valleys in workload, but getting burned out is bad for everyone: the employee, the employer, and the customer. An unsustainable pace for too long will increase technical mistakes and increase technical debt. Push back if an unsustainable peak duration is creating a bubble that will pop – remember the “irrational exuberance” in the stock market crash of 2008?

We are playing the long game. Battles will come and go, but this is about winning the war. Don’t mistake a battle for the war. This means we may need to take some hits in the short term.

Build for elegance and craftsmanship and simplicity. Then it becomes stable, fast, intuitive, etc. In our organization, the goal is to not finish quickly, the goal is to finish at a good place. If you need more than one attempt to finish at a good place, do it. Obviously, if you can finish at a good place quickly, that would be nice, but quick is not the primary goal.

Be totally honest. If there is something not right that is lurking below the surface, bring it to the surface so it can be seen and resolved. Otherwise it may come back and bite us at a bad time as a surprise. If something isn’t right, do or say something. “What are YOU (not someone else) going to do about it?”

Sense and respond. Once you are on a path, consistently look around and see what is going on. And compare that to your decision to originally get on this path, because you may find that at this new point that another path makes more sense because now you have new information. If a change makes sense, then change. As the saying goes, “the first step in getting out of a hole is to put down the shovel”.

Everything that does something for you also does something to you. (I came up with this quote.) That cool third-party library that helps you get something done easier, will also place limitations on you. Or that business partnership that that you outsource to can become a dependency over which you don’t have control. Be careful.

When something is intended for production, get it to be correct before the launch. It gets way more difficult afterwards. Once a customer starts to rely on something, even though it was a temporary hack, it becomes very difficult to pull it back. It will be a better experience for everyone to do it correctly before production.

Don’t accumulate technical debt. Time-slice in debt reduction. If you aren’t making explicit debt payments in each release, you’re probably accumulating debt.

Have a constant sense of urgency. But don’t go so fast that you make mistakes or take shortcuts. Remember the classic horror movie scene where the guy with the chainsaw is very slowly lumbering toward the victim, but the victim is either screaming and holding still, or trying to run away so frantically that they are tripping over themselves and still going to get caught by the slowpoke lumbering murderer? Honestly, if they would just calm down and walk away at a decent pace everything would be fine.  Sometimes you need to slow down to go faster.

We’re not paid to know everything, we’re paid to figure things out. It’s not reasonable to expect a developer to know everything. It would certainly be cool if we did, but in today’s complex world that is pragmatically impossible. However, our value as developers is to efficiently figure things out and then translate that understanding into an implemented deliverable.

It is not an admission of failure to ask someone a question, and you will not be looked down upon. I want folks to be committed to the team (not just to themselves), and have empathy for others that is a source of helping. If someone comes to you with a [let’s assume reasonable] question, would you be offended or look down on them? Of course not. So they shouldn’t do the same to you. You can also think about it this way: You are a member of a team. The goal is not only for the individual to succeed, but for the team to succeed. If you have more to do than one person can handle and get overwhelmed, ask for help. Asking for help is not a sign of your weakness or failure, it is a sign of you caring for the success of the team.

That’s my list for now. There probably will be updates as new lessons are learned and articulated.

tech: Google Play store updates and Android foreground notifications

In the office we’re working on an Android app which isn’t your usual app: we work with a handset OEM to get it pre-loaded on their handsets in the system partition. At the time it required system permissions and had a foreground notification.

The slow rate of ROM OTA updates from OEMs being what they are, we wanted a faster way to deploy updates to our app, so the obvious choice is to also publish our app into the Play store and let Android’s Play services automatically update it from there. Except for one problem: when we published APK updates to the Play store, the handsets would never automatically download the update and install it. Try as we might to debug that, we couldn’t figure out why our app wasn’t getting updated by Play Services like all the other apps. We even got Google to investigate a bit, but never got a solution. So we ended up building a service into our app that would monitor for an updated version being available (via separate metadata that we uploaded to a server) that would then notify the user of the update being available, and then expecting the user to manually click on that to download the update. Yuck.

Fast forward a while, and we rewrite our app so it no longer needs Android system permissions. And Android provides new APIs so we don’t need to use a foreground notification. It still is signed by the same key. As soon as we publish the 2nd version without the foreground notification to the Play store, we notice that the Play services update is working.

So that leads us to believe that when an Android app has a foreground notification running, Play services won’t update it automatically, because it is still running and Play services doesn’t want to update a running app. I don’t expect many developers are using long-running foreground notifications, but FYI for those that are.

While that foreground notification is present, your app is running, and it looks like Play services won’t upgrade your app while it is running in the foreground.

life: the epic road trip

Stats:

  • duration: 23 consecutive days
  • driving distance: 6747 miles
  • longest single day: 856 miles, 15 hours driving (not including breaks)
  • total hours driving: 112
  • average mpg: 29.1
  • gas consumed: 321 gallons
  • number of kids upon departure: 5
  • number of kids upon return: 3 (2 left at college)
  • family groups visited: 9

Monuments / Parks / Museums visited:

  • Frazier History Museum: Louisville KY
  • Gateway Arch: St Louis MO
  • Keeper of the Plains plaza: Wichita KS
  • Mesa Verde National Park: Cortez CO
  • Canyon De Chelly National Monument: Chinle AZ
  • Inter-Tribal Indian Ceremonial: Gallup NM
  • Petrified Forest National Park: Holbrook AZ
  • Meteor Crater: Winslow AZ
  • Historic Route 66: Winona & Williams AZ
  • Grand Canyon National Park: Grand Canyon AZ
  • Cedar Breaks National Monument & Brian Head Mountain: Brian Head UT
  • Bryce Canyon National Park: Bryce Canyon UT
  • Capital Reef National Park: Torrey UT
  • Goblin Valley State Park + Little Wild Horse slot canyon: Hanksville UT
  • Arches National Park: Moab UT
  • Colorado River rafting: Moab UT
  • Mountain biking + horse ride: Moab UT
  • Mormon Tabernacle Choir broadcast: Salt Lake City UT
  • Field Museum: Chicago IL

My favorite moments:

  • Standing at the top of Brian Head peak, 11307 ft elevation, 53 degrees fahrenheit on a sunny August day (not including a significant wind chill), able to see for 100+ miles in every direction.
  • Hiking across amazing landscapes.
  • Laying on a blanket on a cloudless moonless night in Arches NP looking at the stars.
  • Enjoying the dry air (low humidity). It really does make a difference.
  • Driving on a section of historic Route 66 while the kids watched Pixar’s “Cars” movie, and in the end credits the song “Route 66” played while we were driving through “don’t-forget” Winona Arizona.

We flew my oldest daughter home from college so she could do the trip with us. I’m thinking it may be the last time we have all the kids together for a big trip like this. Even after being in the minivan for 3 and a half weeks, at the end I told my wife that I could keep going if there was no need for me to have a job and earn money. The kids did really well. It was work, but it was fun, and we saw amazing sights, and I got to spend quality time with my family.

The following pictures were taken with my phone, and are untouched.

Near the peak of Brian Head mountain
Sunset clouds over the resort at Brian Head
Waiting for dinner to be served at a restaurant after a day of driving and hiking.

life: going to college with financial responsibility

Now that my second child is on their way to college, I think my wife and I have narrowed in on an approach to higher education that is sustainable. Here are the principles we’ve been teaching and doing with our children.

Go to a school that is affordable.

It would be great fun to drive a Ferrari 458. Really great fun. But until I win the lottery, it’s just not realistic to own a $250,000 car. Instead I drive a low-end Acura. The Ferrari is literally 10 times the cost of my Acura. Does that make it 10 times better to drive during my daily commute? The Acura serves me very well. There are way better things to spend my limited money on. The same concept applies to schools. Is the small class size and name and some extra networking opportunities really 10 times better than a qualified school with in-state tuition? Perhaps better, but 10 times better? I think you have better things to spend your limited money on. If you can get scholarships or tuition reduction, then go for it. If your money is unlimited, then go for it. Want to take out loans for the expensive school? Read the next item.

Do not go into debt.

I hear people talking about exiting school with a degree and 50 or 100 or 200 thousand dollars in debt. I find that crazy. It’s a mortgage that you can’t live in that will take 30 years to pay off. Do you really want to burden your existing or future family with that much debt? It does not have to be that way. Pay as you go. If you need to take a semester or a year off of school to earn money to carry you through the next period, do that. There is absolutely no shame in that. You need to go at the right pace so you exit school with zero debt. And don’t go into a crazy-expensive private school – there is no shame in a public university.

As parents, we will match whatever you can get.

I can help because it is expensive, but I want you to have financial skin in the game. So we will do a 1-for-1 match of whatever income or cost offset you can get. Get a job and earn some money, and we’ll kick in a matching amount. Play video games and hang with your friends and earn nothing, and we’ll kick in nothing. Now do you have motivation to put it in that scholarship application? My first daughter is earning money to cover her living expenses, and we pay for her tuition, these being roughly equal. I like that arrangement, as she gets quicker feedback on managing her living expenses, those being more variable than her tuition expenses – she learns budgeting and balance.

As parents, we will cover your first semester or year expenses while you get your feet under you.

It can be a tough transition from high school to college. When you get to college you are going to need to apply yourself at a substantially different level than that last year of high school where you breezed through your classes and had a case of senioritis. Although ultimately I want you to be financially responsible for your education, more importantly I want you to be successful in your education. In the beginning I want the transition to college to be successful. I don’t want the first year or semester to be a failure when you combine tougher school with a job, which may also be the first time you’ve done this combination. So focus on school for the first semester or year, figure it out, and I will cover your tuition and living expenses. I’m not going to cover your playing expenses: learn to be frugal and figure that out on your own.

Although we are helping financially, you are responsible for getting the bills paid.

Is tuition or rent or meal plan payment due? I’ll transfer my contribution to your bank account. But you need to go to the tuition office and write the check or set up the draft, make sure the funds are in the account, and keep track of the due date and get it in on time. That’s just part of being an adult. And you’ll be doing these kinds of things the rest of your life, so you better know how to do it now, because they are really important.

I don’t want to sound judgmental, but having parents pay for everything and the kids not having any responsibility was not how we wanted to do it. I love my kids and I want to help them grow, be responsible, and independent. This is part of our path in getting there.

food: the North Carolina Historic Barbecue Trail

I stopped by a recommended BBQ restaurant recently (Hersey’s in Graham NC), and noticed a plaque on the wall. Part of the plaque reads:

NCBS (the North Carolina Barbecue Society) has designed a barbecue trail from Eastern North Carolina to Tennessee. The trail will meander across the state with stops at 24 NCBS Historic Barbecue Pits. These pits were carefully and selectively chosen by the NCBS board as representatives of the distinctive methods and barbecue cooking styles that have made North Carolina the Barbecue Capital of the World. Each NCBS Historic Barbecue Pit still cooks the old fashioned pit cooked method. And each NCBS Historic Barbecue Pit will proudly display a specially designed emblem depicting old style barbecue cooking that is part of the tradition, heritage and culture that NCBS seeks to preserve.

This is for real. And when I got home I looked it up. Yup, there is a map that meanders through the state, with a cluster of sites in the center of the state. Time for a road trip!

life: visiting Disney World?

My family visited Disney World over the Christmas holiday. Yes, this is the absolute peak season for visitors. So I went bracing myself for an hour in every line all day long. Right before leaving we talked to a neighbor who recommended a service named RideMax. He told me you feed it a list of things you want to see at the park, and it optimizes a plan for what to see when. I said we’d give it a try.

So we did give it try. We spent a 4 days at the parks, and we never spent more than 15 minutes in a line. Again, this was during the peak season. The wait times we saw posted as we walked by other rides were between 30 and 120 minutes, most were an hour or more. We paid $15 for our RideMax subscription, but I would have paid $100 for it with the value we got out of it with 8 people for 4 days. We were totally sold on how it works. I assume it sends us to where the crowds aren’t. We wandered off the plan once, and paid for it with a 90 minute line – we learned to stay on the plan. We got to see everything we wanted to, it just gave us an order and a time for each. We even got done with our plan a bit early. We were able to modify our plan from our iPhone.

If you’re going to one of the Disney parks during a congested time, I highly recommend RideMax. Personally, I won’t go without it.

life: family guidelines

Several months ago my wife and I were lamenting some of the things we saw weren’t going well in our family. So after some long talks, we came up with a set of guidelines for our family that really hits the nail on the head (at least for the issues our family is struggling with). There are three parts:

Love, Respect, Kindness. The bickering, mean voices, not sharing, and general selfishness are things we do not want in our home. It really sucks the spirit out when these things are going on. And it’s not right to be nice to strangers and mean to your own family. Close quarters does uncover the flaws in people, but we are all flawed and we need to deal with it. The attitude we have can make all the difference.

Do Your Duty. It’s not fair to have 5 capable people making a mess of the house and not cleaning up after themselves, leaving mommy to do it all. Do your homework before your recreation. Many hands make a light and even load.

Play Together. We need to have fun, and we need to do some of it together instead of in isolation. We need to budget time and money for family activities, and do things we can all participate in, whether it is a trip somewhere for a couple days, a handful of milkshakes, or a board game on a Sunday afternoon. We need to have positive interaction together and build memories.

I share this in the hope that someone else might benefit from what we’ve learned, and to see if there are suggestions from others.

life: extending the life of a household water heater

My home has a gas-powered hot water heater located in the walk-up attic (3rd floor). I’d like to give the architect a dope-slap for doing that. Especially when the master bath directly below the water heater takes the longest to receive the hot water. But the real reason why the attic location is an issue is because there have been four incidents where the water heater has leaked which ended up sending water spilling down the lower two floors. If only they could have put it in the garage. Sigh.

Anyway, one of the leaking incidents occurred recently. I just woke up the kids for school. I’m standing in their bedroom on the 2nd floor and asking myself, “why do I hear water running above me?” So I grab a flashlight and dash upstairs to the attic. There is water flowing out of the top of the water heater and down the sides, as if it has ruptured. At least the water heater is surrounded by a drip pan which drains outside the house. The drip pan was full to the brim, and just a little bit had spilled out. I quickly shut off the inlet valve on the water heater and the pan continued to drain. It looked like less than half a gallon had spilled outside of the pan, I consider myself very lucky. Time to call for a replacement water heater. I also take note to make sure the drain on the outside of the house for the drip pan is free of debris that may interfere with draining.

Since the water heater is in an attic closet bounded by the A-frame roof, there is limited height. It turns out that my water heater is a few inches shorter than the typical short model, so it is a bit of a specialty product that the typical big-box retailers don’t carry. So I need to call a real plumber. Although the plumber did a first-class job and was done within 4 hours of when I called them, it also cost me $1100. Ouch.

So while the plumber was working, I was my usual inquisitive self and was asking questions. And I had done some research on the net before calling the plumber. The typical lifespan of a water heater is about 8 years. Mine had died at about the 9 year mark, so it wasn’t out of bounds. The most common cause of water heater death is a build-up of sediment at the bottom of the tank. This sediment is present in the water supply, and simply settles while in the heater. There is a relatively easy way to flush out the sediment, which until then I had never heard of. Here is what the plumber told me:

The water heater should have on it what looks like a spigot near the bottom where you can attach a garden hose, just like the hose bib on the outside of your house you use to wash your car and water your garden. And once a year, you should do just that – a power flush. Get a garden hose (make sure it doesn’t leak before you do this), connect it to the spigot on the water heater, run the hose to a safe location (ie, a bath tub or out a window to outside), open the spigot on the heater and let the water blast though the hose for about 5 minutes. The water will be hot, so be careful that the hose and water output doesn’t hurt anyone or anything. Do not shutoff the inlet of the water heater, you want the water to blast out of the garden hose at pressure instead of simply (partially) draining the tank without pressure. The reason you want it to be pressurized is because of what is behind the spigot – a tube that causes the water flow to stir up the sediment on the bottom of the tank so it can be flushed out the spigot. When the 5 minutes are up, close the spigot (make sure it is fully closed), and remove the garden hose. Be careful of hot water that may still be in the garden hose while you are removing it.

You should do this once a year. I got a Sharpie pen and wrote on my new water heater “flush on March 15”. Had I known about this earlier, I would have done it and would expect a longer lifetime of my water heater. The plumber said that it should be possible to get several more years than average from a water heater that is well-maintained.

life: advice for the new father

After having significantly more kids than the average family (well, I didn’t give birth to them, but I did cut the cord), to say that I have learned some things is a vast understatement. Some of these things took a long time to learn, even as late as the last kid. For the men out there that are a first-time dad, here are some of the lessons I learned.

  • Having kids is the most wonderful thing that can happen to you (outside of getting married to a wonderful woman, of course). Having kids will also be the most stressful and trying thing that can happen to you, which hopefully is unlike what happens being married to a wonderful woman. Let me be blunt: having kids will force you to give up selfishness. You are going to have to put more things than you are used to of your own desire on the back burner. Prepare yourself for doing that, and do it willingly, and do it out of love for your family. Guys typically aren’t naturally good at this. You aren’t supposed to completely sacrifice yourself, especially in a passive-aggressive manner, but you are going to need to give up things. If you can be selfless, and do it for the right reason, you will be amazed at the happiness it brings to you and your family. Not giving yourself to your family is going to put your wife through the wringer, don’t do that to her.
  • If you wife has a vaginal birth (not a C-section), you are going to need to give her time to heal before being physically intimate again. Being intimate will happen again, especially if you do the bullet above. However, recognize that pushing a small watermelon through your pelvis is going to do some damage. If you listen to the OB, they will probably give you an estimate on how long your wife will need to heal after birth. I suggest that you double it. Putting your wife in pain for your satisfaction is not consistent with the spirit of physical intimacy, no matter that you are not used to waiting that long. Be patient and deal with it. If men were the ones to get pregnant and give birth, we would go extinct as a species.
  • I was talking to a guy several years ago, and he said, “When I have kids, I will work at my employer as long as possible every day so I can afford to buy my kids all the stuff I didn’t have.” My response was, “Dude, kids (well, the small ones until we teach them otherwise) don’t care about objects. What they want is your time. They are happy playing with a big cardboard box, and they want you to get on the floor and play with them and read them stories. They don’t need things, they want you.” The best present you can give your kids is yourself when they ask for it. That doesn’t always correspond to when you want, or what you expect. When you depart from this mortal life, your family is the only thing you can take with you. The sad thing is that at some point your kids will probably stop asking to play with you.
  • It is so easy for your wife to get completely consumed with being a mom. With a newborn, there is not a full night’s sleep and there are no vacation days.
    • You have to give her a break. When you come home from work, take over the kids and let your wife do something of her own choosing for a couple hours. It doesn’t matter if you had a hard day at work and need some break time, she started when she woke up in the morning and won’t finish until she goes to bed, and likely there will be middle-of-the-night fulfillments. That’s more than your 8.5 hours.
    • The rule is “when the baby sleeps, mommy sleeps.” Baby naps are not always to be used for mom to catch up on chores. Mommy needs to catch up on sleep so she can be a functioning person. When she is caught up on sleep, then she can catch up on chores.
    • Don’t forget that your wife is still your wife and not just a mom. It is very easy for both parents to become parents and forget to be spouses. Find someone you trust to be a babysitter earlier than you think. Go out for a short date and talk about things other than just the kids. The intention is for the kids to grow up and leave the house, but your wife should stay with you.
    • The house is going to be a mess. Just expect it. Instead of being part of the problem, be part of the solution. You will need to own more chores. There is no sharing equally and keeping track of points. You need to jump in and do as much as you can, independent of what your wife may or may not have done.
  • If your wife is breast feeding instead of using formula, her breasts will get bigger. However, they also become very tender, especially at the start of nursing. Treat her carefully. The skin will toughen up over a few weeks to handle the nursing. They will return to usual size after nursing stops. Also, when a baby is breast feeding, the poop actually isn’t too bad. It’s when you go to formula or solid foods that it turns into the classic stinky mess.
  • Be patient. Take a deep breath and don’t have a temper. Kids will throw enough temper tantrums, you don’t need the parents doing it too. Just let the unimportant things go. Yes, having kids will dramatically alter your perspective on what is important. Lack of patience is one of the easiest ways to mess up a family.
  • Jump in and be willing to figure it out. Every child is different, and every parenting situation is different. Figure out what works for you and your family. Experiment. What works for child #1 may not work for child #2, even if they are twins. You’ll find lots of things that don’t work, followed by an “aha!” moment.
  • The What to Expect When You’re Expecting and What to Expect the First Year are good books that dads should follow along in as their wife reads. At lot of stuff that seems weird actually is quite normal. And the only way to learn it is by experience.
  • Kids don’t actually cost very much money, except perhaps in healthcare. If you can get hand-me-down clothes or go to Goodwill (they destroy clothes anyway), it’s really not bad. They don’t eat much, don’t need expensive toys, can travel in your lap on airplanes, etc. Don’t bother buying them nice new expensive things, because when they are little they don’t care. It’s later when they need car insurance and college tuition that it starts to hurt your wallet.
  • Kids are amazingly resilient. They bounce good. Typically the parents need to take a deep breath instead of panicking.

I hope you find this useful. Having kids is a trip. I didn’t think it would be so before it happened, but my life would be so empty if I lost my kids. My life is way more complicated now with kids, but I’ve grown to prefer it that way – it’s more fulfilling.

life: confessions of a LASIK patient

It’s not something I was originally looking to do. We had been using our pre-tax health savings account for the kids’ braces, and miscalculated the payment schedule, so we are at the end of the year with $2000 of unused money in that account. Use it or lose it. (I checked, when unused the money reverts back to the employer, not the government.) So my wife said “Aha! I want to get LASIK.” The procedure costs about $2200 per eye, so our health savings account would cover about half of the total cost. Our assumption is that it is better to spend another $2000 on something you want rather than throw the original $2000 away (we may need to rethink that one). She went in for a consultation, and discovered that she isn’t a good candidate because her cornea is too thin for the procedure. For her vision to get corrected, they were recommending a corneal implant, but it would be twice the cost and it sounded more risky. So she nixed the idea of corrective surgery. So she says to me, “Hey honey, how about if you take a look at LASIK?” I’m a bit uncomfortable with the idea of getting corrective surgery, since my career is sitting in front of a computer screen all day, and my contacts and glasses work fine, I’m worried about long-term complications. Losing my eyes would lose my career. Since getting an initial consultation has no cost and no commitment, might as well give that a shot. So I make an appointment.

After a number of tests, the doctor tells me I’m a good candidate. My uncorrected vision is 20/200 in one eye, and a bit worse in the other eye. However, with my glasses I can see 20/15. I do have some astigmatism that make my contacts more like 20/20. One of the doctor office people goes into detail explaining the procedure and such. Later I ask a friend at church who is an eye surgeon in the military about the procedure and the equipment that this office uses, he says he has performed about 60 of these procedures on patients, and had it performed on himself about 2 years ago, and he feels confident with it. He recognizes the equipment at this clinic (VISX Star S4 IR Laser / iLASIK / WaveScan / CustomVue) and says it is widely used and pretty much state of the art. So I’m thinking that if a doctor in this field would do it to themself, that is basically the ultimate test of confidence.

The office has an appointment open for the following week to perform the procedure, so I take it.

I’m asked to not wear my soft contacts for a week before the procedure. I’m told it has something to do with affecting the shape of the cornea, and they want it to be stable for the procedure. So I wear my glasses for a week (even during a touch football game that week).

There are two lasers that will be used for the procedure. The first, the intralase, creates the corneal flap. The second, the excimer laser, reshapes the cornea after the flap has been peeled back.

On the day of the procedure I show up at the office at 8am. No need for fasting, since there is no general anesthesia. They take me into a room for another test, they kind where you sit in a chair and put your chin and forehead in front of a machine that looks in your eye. It’s a scanner that measures my eye so that the excimer laser knows what to do. The office’s brochure says “…the VISX WaveScan creates a WavePrint map, which reveals the way your entire optical system processes light. This creates a personalized fingerprint of your vision…allowing each of your eyes to be treated for their unique imperfections.” Relax and look at the red dot.

Then I’m taken to a waiting room. I’m given an optional Valium dose and some consent / waiver papers to sign. I read the papers and I see a few things that surprise me. First is verbage about getting one eye done at a time, therefore if something goes wrong, at least you still have one good eye. So it asks for me to write a short explanation why I want to have both eyes treated at the same time, as if doing so is a bit unusual. Funny, I don’t remember the doctor explaining that as an option, but my memory isn’t perfect. Second, there is some verbage explaining the risks of the procedure which includes terms like “potential catastrophic loss of vision”. That gives me pause. I had asked before quite a bit about the risks and the answers all sounded reasonable. So I assume this verbage is the cover-your-butt legalese (“you acknowledged that it was possible for this to go wrong”) and sign it. They give me several doses of anti-bacterial drops and a dose of numbing drops.

I am given a surgical cap to match my street clothes and proceed to the intralase room. The intralase system shines a laser to create small air bubbles in the cornea. Once the bubbles are there, the corneal flap can be peeled back without the need for a cutting knife. The staff tells me that the nice thing about the air bubble approach is that if the doctor doesn’t like how the air bubbles have formed, the rest of the procedure can be aborted and the air bubbles will dissipate on their own within a day or so. I lie down on a table (reclined chair, actually) and am given more numbing drops. They then bring out a device that looks like a small magnifying glass with the handle. The lens of the magnifying-glass thing is small enough fit on the surface of my eye, which is what they do. It doesn’t hurt, and I can see through it. Then I hear the doctor tell the technician, “Apply suction.” Then I feel this magnifying glass thing start sucking itself firmly onto my eye, and the vision in that eye goes dark. It’s hard to describe other than being visually numb in that eye. But the eye isn’t entirely physically numb, I feel the suction. The suction was a bit uncomfortable, even for someone used to contact lenses. Then a good-sized machine is placed over me that looks like a digital camera that is 3 feet wide, with the lens right over my suctioned eye. This is the intralase laser. Then the lens of the intralase laser then is slowly lowered until it makes contact with and is lined up to the magnifying glass on my eye. But then it keeps going, pushing down on my eye and creating pressure. It creates quite a bit of pressure, or in technical terms, it squishes the heck out of my eye. Since I’m laying on a table, I’ve got nowhere to back up. I let out a gentle verbal “aaaaaah” to let the staff know I’m uncomfortable (although not in significant pain) and the doctor says “Yes, the pressure may be uncomfortable, it won’t last long.” After about 30 seconds of squishing the machine lets up. The pressure was such a distraction I didn’t even notice the intralse laser working. They release the suction on the magnifying glass thing and I let out a sigh of relief. They repeat the process on the other eye.

After the intralase is done with the second eye I can see, my vision is like looking through glasses that have been smeared with Vaseline. I assume I’m looking through all the air bubbles that were just created. They lead me down the hall to the room with the excimer laser. I lie down on the table (reclined chair). They place a shield over one eye and start working on the other. They start by taping back my upper and lower lid. They then place what looks like a round grommet in the eye, I’m assuming it is to prevent the lids from interfering with the corneal area. About 2 feet in front of my face is the excimer laser system, it has an illuminated white circle and a flashing red dot in the middle. It’s kind of like an electronic Cyclops. I can see the doctor gently poking around in my eye with a small instrument. After a few moments of that, the Cyclops becomes more blurry that it was originally. I assume at this point that the corneal flap has been peeled back. They ask me to focus on the red blinking light. Another few moments and then I can hear the staff saying words like “acquiring…verifying…25 seconds…” Then I can hear them counting down and the laser is making a loud clicking noise. Midway through this there is a difference in how focused that the Cyclops appears. The clicking stops and then the doctor does some more gentle poking around in my eye with an instrument. In a moment the Cyclops comes into better focus, and I assume the corneal flap has been placed back. The grommet is removed, the tape is removed, and they move to the other eye. After the numbing drops I received earlier, and squishing of the intralase system, I don’t really feel a thing during this entire excimer process. As far as the discomfort goes, if you can survive the intralase process, you are home free. The tape removal (with a couple of my eyebrow hairs) is what hurts the most in the excimer room.

Then I am led to a regular exam room and the doctor uses a regular non-invasive microscope to look at how the corneal flap is resting. He says “Perfect.” And I’m done. They lead me out to the waiting room for my wife to take me home. Since the air bubbles are still present, my vision is too cloudy to drive myself, and there is a bit of discomfort in my eyes. It feels best to leave them closed. But I can open them enough to dial my cell phone. They give me a packet that includes two kinds of medicated drops, and a package of lubricating drops. There is more stuff in the packet which I’ll describe below.

My wife takes me home. I can see a bit, but I prefer not to keep my eyes open. I lie down in bed to just rest, turn on my iPod, and the Valium in my system helps me take a 2 hour nap. I’m supposed to take the medicated drops every 2 hours, so I do that after I wake up. The drops don’t hurt. I get a bite to eat. Now my eyes are starting to hurt, as the numbing drops wear off, and I definitely prefer to keep my eyes closed. I lie back down in bed and put my iPod back on. I take another unanticipated 2 hour nap, I’m assuming it’s the Valium still in my system. My eyes feel like there is some sand in them but not scratchy. More drops, this time I stay awake through the podcast I was listening to earlier. About 6 hours after the surgery, the discomfort starts to get better. Keep putting in drops every 2 hours. I still prefer to keep them closed, and I figure out part of it is light sensitivity. If I cover my head with a blanket so it is dark, I can open them longer than in the light. Still hurts a bit.

Some friends come over that evening to check on me. I can see them and walk around the house by myself, although there is still some cloudiness. One of them says, “Wow, I was expecting to see you with giant bandages on your head, and instead no bandages and you can even see me.” Considering that 8 hours earlier my eye was peeled open and shot with a laser, it is pretty amazing. I get a good night rest, even with the day naps. For the first week, I need to wear sports goggles while sleeping at night. I assume it is for side sleepers so the pillow doesn’t put pressure on the eye, or to prevent you from rubbing your eye and displace the corneal flap.

The next day the sandy discomfort is gone. I can see with good focus but with what I’d call a “halo effect”. Any areas of high contrast appear to have a fuzzy halo around the bright part. It’s almost like I have my glasses with a little bit of fog. I go to the doctor’s office for a followup visit. The vision in both eyes is checked with the usual chart and I can see 20/15 in both eyes, even though I still have the halo effect. I ask the doctor about the halo effect, and he says it is normal and will probably take about 3 months to clear up. I work at home using my laptop most of this day. I’m back in the office the next day.

A few days later, the cloudiness has cleared up but the halo effect is still there. I’m starting to get used to the halo a bit. The halo does cause some difficulty for night driving, but it is manageable if I am careful where to look and concentrate. I also have an increased need to wear sunglasses in daylight.

It feels quite weird to not take my contacts out at night and instead go to bed with clear vision and wake up with clear vision. I do feel a tiny bit of pressure around my eye, but perhaps that is from the cold I have. A week after the procedure I should be able to graduate from the medicated eye drops, but I’m told I should use the lubricating eye drops for at least a couple more months. So far so good. I’ll consider it a real success when the halo effect goes away and I don’t need eye drops anymore.

In retrospect, there wasn’t much pain or discomfort.

You might want to take a look at Wikipedia, they have a pretty good article on the subject.

I’ll post followup comments here.

fun: recipe for cinnamon rolls

Almost as good as that expensive store in the mall, which I won’t name. These are less expensive and you don’t need to stop eating at one. There is a recipe in our house that I’ve been playing with. So far, everyone loves these and they are reasonably easy, especially if you have a bread machine.

I have noticed that there is a difference between butter and margarine. Using butter in the bread dough gives it a delicate texture without being crusty greasy that margarine does. But using margarine in the filling and frosting makes it more sweet than butter. So I selected margarine and butter in specific places below. Or if you want you can go all butter.

Plan ahead to take the margarine and butter (all 3 sticks) and cream cheese out of the fridge in plenty of time to warm to room temperature before using.

BTW, “tsp” means teaspoon and “tbsp” means tablespoon. Read carefully.

Bread machine dough:

  • 1 cup water
  • 4 cups bread flour
  • 1/3 cup sugar
  • 1 3/4 tsp salt
  • 1 stick butter (room temperature, slightly mushy)
  • 1/4 cup dry milk
  • 2 eggs
  • 2 tsp yeast

If you are not letting this sit uncooked for a long time, you can substitute the water and dry milk for 1 cup of regular (wet) milk, if that is easier for you.

Dump all the above items at once into the bread machine and run the dough cycle (mix and knead, not bake). Prepare a flat surface like a clean countertop where you can roll the dough. You’ll want to flour-dust the surface so the dough doesn’t stick to it. Take the dough out of the machine and roll it out. Aim for a rolled-out rectangle of bread dough that is approximately 12″ x 24″. It’s OK if it isn’t a perfect rectangle.

Filling:
Take a 1/2 stick of margarine or butter (room temperature, slightly mushy) and evenly spread it across the entire dough surface using a spatula or similar. Go all the way to the edges. In a bowl mix 2 cups brown sugar and 3 tbsp cinnamon (ground). Brown sugar works much better than white granulated sugar, and common ground cinnamon is fine. Use a fork to mix them together in the bowl and break up the brown sugar lumps by squishing them. Dump the mixture on the buttered dough surface and evenly spread it around the entire surface with a spatula or similar, all the way to the edges.

Roll up the dough, using the long side of the rectangle: the roll should be 24″ long when complete. Keep the roll reasonably tight instead of saggy. When done, you should have a roll that is 24″ long and about 3″ in diameter. Using a steak knife or other good cutting instrument that cuts without too much squishing or ripping, gently cut the roll into pieces about 1 3/4 inches wide. You may wish to discard the two end pieces you cut from the roll, because they are uneven, but only if you are a perfectionist. Otherwise dunk them in cinnamon sugar and put them in the baking pan too. I would recommend a spray of Pam on the baking pan, even though there are already 2 1/2 sticks of butter present. Gently place all the cut pieces into a baking pan on their flat cut side. When placing in the pan, try to avoid letting the cinnamon sugar filling fall out. The rolls should be spaced out in the pan. It should fill about a 9×9 pans plus a 9×13 pan. Give the dough time to rise, so that the rolls in the pan are starting to touch each other. Placing the pans in a 150 degree oven may help the rising process if you are in a hurry. Cover the pan while rising to keep the dough from prematurely drying.

If the rolls fail to rise, then the filling will run out and pool on the bottom of the pan during cooking. If this happens, take the rolls out of the pan quickly after they are done cooking, then the pooled filling while it is still soft can be scraped off the bottom of the pan and mushed on the top of the rolls. If the pooled filling cools in the pan, then it acts like dried epoxy. It’s better to budget at least a couple hours for rising so you don’t get pooled filling.

Bake the rolls at about 385 degrees until the the bread dough on top starts to show a bit of toasty brown. You don’t want them to overcook, as the bread could turn dry. You want them to stay a bit moist, with a very slight hint of dough. Take them out of the oven to cool a bit. Let them cool inside the cooking pans, they will soak up some of the melted filling. While they are cooling, work on the frosting.

Frosting:

  • 4 oz of cream cheese (room temperature, half of a standard 8 oz package)
  • 1 stick of margarine/butter
  • 1 1/2 cup powdered sugar
  • 1 tsp vanilla extract
  • 1/8 tsp salt

Put all these ingredients into a bowl and blend with a motorized mixer until it changes from dusty clumpy to smooth moist frosting. You might want to add a bit more vanilla extract for some more flavor.

After the rolls go from hot to warm, apply the frosting. If you put the frosting on while hot, the frosting will melt completely, which probably isn’t what you want. Eat them soon after applying the frosting, while they are still warm. All of them, which is why some friends or neighbors should be present or within delivery distance. This ain’t health food, it’s happy food.

personal finance: credit bureau freeze

Be careful of how much debt you carry. A mortgage is OK, but creditors will generally give you a bigger loan than you can really afford. An automobile loan is OK, but recognize that an automobile can depreciate faster than you can pay the loan – it sure isn’t an investment, it’s pure expense. One credit card is OK for purchase convenience, not carrying a balance. If you have all three of these, you don’t need any more. I’m tired of getting pre-approved offers in the mail literally twice a week all year long.

Also be careful of how much personally identifiable information you disclose. If a web site asks for my birthday for marketing purposes, I won’t give it. If they require it, I will give a fake one. If you’ve talked to anyone hit by identity theft, you don’t want to experience it yourself.

For these purposes, I have asked the credit bureaus to freeze my personal information so it isn’t disclosed to lenders. Since lenders need to know my credit worthiness before approving credit, this means no credit can be opened in my name. Right now I have all the credit I need and don’t need more. This also prevents someone else from obtaining credit in my name without my approval, i.e. identity theft. If I do need to get more credit, I can unfreeze just long enough for the new lender to approve me, then it goes back to a frozen state.

Clark Howard has some good pointers on how to freeze your credit information. You need to do it with each of the three credit reporting bureaus. All it takes is a letter and $10 for each one. Clark even provides the form letter. My requests were processed in about one week. Frankly, I recommend everyone do this. Recognize that the credit bureaus are in the business of selling your credit information to lenders, and lenders are in the business of getting interest and fees from you. Nobody is in business for you except yourself.

You also are entitled to a yearly credit report for free. Get your free report each year to make sure it is accurate. Then when you need credit, there won’t be any surprises. Start with the US Federal Trade Commision web site to make sure you go to the right site that offers the free credit report (there are copycats/imposters out there that may require costs).

Keep your finances in order and you’ll avoid a lot of pain.

cars: replacing the dash lights in a Honda Odyssey

I have a 2000 Honda Odyssey, and although it has been a wonderful vehicle, it is at the age where the dash lights are starting to burn out. So my clock backlight has failed, along with the lighting of several buttons on the dash. When I went to get parts at the dealer to replace the clock backlight, it turns out that the person in front of me in line at the parts department was there for exactly the same reason for the same model. It’s relatively easy to do, if you know where to start. That’s what I’ll explain here. I started taking the dash apart to access the clock backlight, but you’ll get to the other lights along the way (except the instrument cluster).

First, the bad news. The clock doesn’t pop out forward from the dash. It pops out backwards behind the dash. So don’t try to pop out the clock from the front of the dash, you’ll just end up damaging the dash. This means to access the clock you need to take out 3 large pieces of the dash. The good news is that most everything just pops out. What doesn’t pop out is held in only with standard screws. Once you understand how it works, you’ll say “ah, that wasn’t so bad.”

wedge

The first two major parts of the dash just pop off. It’s not difficult, but it’s not obvious what to do. I started prying next to the steering column and gently worked my way across. I started at similar points on both sides. Then you can slip your fingers behind and gently work it until it pops completely loose. For the prying I used a plastic/vinyl flat tool that happened to come from my kitchen. I would recommend against a hard tool like a screwdriver because you’ll leave marks. The plastic kitchen tool I used left no marks.

right_cables

Once you’ve popped it loose, don’t try pulling on the fascia to get it completely away from the dash. Each piece of fascia has switches with wiring that goes behind the dash – those wires are still connected. The wiring for each of these switches can be disconnected from the switch, and once that is done then you can move the fascia all the way away from the dash. Each wiring connector has a tab that locks it onto the switch, so don’t try disconnecting the wiring by force. Instead you’ll need to find the tab for each connector, depress the tab (they do take a bit of effort), and then the connector should slide off with an unforced pull or a little bit of wiggling. DO NOT pull by the wires; instead you should be pulling by the block connector while depressing the tab. Some wires have more slack than others, so you may find that a certain order of disconnecting/reconnecting is needed.

For the right fascia around the stereo, the shortest wires are for the interior light switch and the fog lights (if installed) – do those two first. Then you’ll have more slack for disconnecting the three wire groups for the climate controls. My van does not have the satellite navigation.

Here is a picture that shows the back of the right fascia, so you can see where the wires connect to.

facia

For the left fascia around the sliding door buttons, the wires were about equal length so just do them in the order that you can reach.

Now the bottom half of the dash on both sides is removed.

left_screws

right_screws

Next comes the top half of the dash. This is all one piece. It is held in by screws along the bottom, and tabs across the top. Get a phillips screwdriver and remove the screws (don’t lose them). Then gently work the top half of the dash out. It is not connected to the instrument cluster (speedometer, etc.), just the border.

clock_cables

Again, be careful of the wires for the clock and hazard blinkers – you must disconnect the wires using the tabs before pulling the fascia all the way away from the dash. Once you’ve gotten this far then you have completed disassembly.

clock_rear

Now that you can access the rear of the clock, the bulb comes out with a quarter turn of a screwdriver. It looks to be the size of an LED, but I think it is a regular but very small bulb. You can also verify it is burned out by using a multimeter to test for continuity. Put the new bulb in (should cost about $3 at the dealer). While you are there, check the rest of the bulbs in the switches and the climate controls. I think there are two different kinds of bulbs: the common one is for backlighting, and the other is for “on” indicators such as the cruise control and fog lights.

For the top half of the dash, position it in the correct place, reconnect the wires, pop it in, and replace the screws. Do the same thing for the left and right fascia, except that there aren’t any screws. Just remember to reconnect all the wires, or some things will stop functioning.

You’re all done! See, that wasn’t so bad. Don’t you feel proud that you didn’t pay the dealer $90 in labor?

mosquito bites: getting rid of the itch

I live in an area where mosquitoes are common. You don’t want to be outside at dusk without repellent, or you will get eaten up. And I hate mosquito bites and the itches. A bite will bother me for 2-3 days. I am better at dealing with pain than itches. Thank goodness for DEET to keep the nasty critters away. In the event that repellent doesn’t work or I didn’t put it on, there is another alternative: After Bite.

After Bite is a liquid that you put directly on the bite after it has occurred. It reduces the itching. In my opinion, it really reduces the itching, depending how soon you put it on. If I notice an itch coming on within a few minutes after getting bit, and I put on a couple applications of After Bite, the itching generally goes away in less than an hour, and the swelling also goes down. This all may sound like a TV infomercial, but the stuff really does work for me, seriously. Of course, don’t scratch the bite area even with this treatment.

From what I can tell, the active ingredient is ammonia, it’s not an anti-histamine like Benadryl or an anti-inflammatory like Cortaid. I don’t think it smells as bad as repellent, and I wouldn’t have guessed that ammonia would have this effect, I can just say that it works for me. I have found After Bite at Eckerd and Target, so I would assume you should be able to find it at similar retailers. In the stores it’s generally located next to the repellents.

Hmm, maybe the dad in My Big Fat Greek Wedding was on to something with the Windex…

tips: the “newspaper” test for decision making

Have a decision to make at the office, and it isn’t obvious what to choose? Usually we look for positive confirmation using tools such as rationalization. But it can be often helpful to try negative confirmation. This approach can open up a whole new world of insight.

Use this handy little tool that I call the newspaper test. First, make a tentative choice, then think of the worst case scenario for that choice, then plug into this template:

  • Headline: [worst case scenario]. [the choice]. What were they thinking?

Let me give you an example:

  • Headline: Lost all the source code for the product, release date delayed indefinitely. They never got around to putting it into the repository before a hard drive crashed. What were they thinking?

The second stage of this test is to pretend that the above item is printed in a newspaper that your coworkers, management, and competition read. Now put yourselves in their shoes, and ponder on how they might react to such a published item. For example, think about what the newspaper article would infer about your decision and what the consequence might be. For example:

  • Infers: ouch.
  • Consequence: Marcel, call for you on line 2 from the VP. She doesn’t sound happy. Do you have your financial affairs in order?

Now the nice thing about this tool is that it works on a range of data. Let’s try another example:

  • Headline: Lurking bug found minutes before they started manufacturing. They ran the testcases just one more time to be sure. What were they thinking?
  • Infers: awesome. This is a person that makes sure things are right and is constantly thinking about customer satisfaction.
  • Consequences: Marcel, this poster-mounted award check for you doesn’t fit through your doorway by myself. Can you give me a hand?

And as an added bonus, this tool also works for personal situations outside of the office. For example:

  • Headline: Mayhem. Father lets 3-year old play inside car with the keys left in ignition. What where they thinking?
  • Infers: typical male.
  • Consequences: Sir, I’m glad nobody was hurt, but I’m still not clear on why we need to dispatch three tow trucks and a crane to your house.

So, the next time you are wondering what choice to make, give this tool a try!

life: Forgiveness

[Note: the following post discusses some Christian principles and has religious references. I have respect for other beliefs and don’t want to offend anyone. If you are offended by Christian references, then you might consider skipping this post.]

Why would anyone want forgiveness?

I found myself each week in a meeting where tucked away on a shelf, not quite out of sight, was a plaque for someone I had known. The person had since moved out of our area, and for whatever reason, I suppose the plaque didn’t get to them in time before their departure. So there it stayed on the shelf. One time long ago I was with this person, just the two of us riding in a car. And there in the car was this uncomfortable silence that I wanted to fill. Not being a conversationalist, I always struggled to fill that silence that bothered me. During one such uncomfortable silence I scraped around in my brain for something to say, and out it came. As soon as it left my mouth I realized I had just inserted my foot, a confidence betrayed. I definitely didn’t have any intention to hurt, but I expected it would. It took a while to realize what I had done, the person never said anything, but I felt awful. That had been several years ago. Everytime I saw that plaque I was reminded of that experience in the car. It was haunting me. I wanted to make that feeling go away. I wanted to be forgiven.

Being human, each of us is going to screw something up, miss opportunities, and hurt people. It is inevitable. It is part of who we are, and part of life. As much as we want it to go away, it will be there, whether we try to ignore it or not.

What does it mean to get forgiveness?

There is a difference between forgiving someone for a wrong commited, versus that person reaping the consequences of their actions. For that person who breaks the law, the victim can forgive, but the perpetrator may still go to jail.

Even after a wrong has been committed, it usually isn’t possible to undo the wrong. Thefts may be returned, but it is not so easy to mend a broken trust, restore virtue or reputation. When I see a person who has just been caught in the wrong, I think “what is done is done. The question is: what are you going to do now?”

Seeking forgiveness is more than just saying you are sorry. Giving forgiveness is more than saying, “yeah, OK”.

How does forgiveness work?

I see it as two kinds of forgiveness: from the victim and from the Lord. I believe it is more than just confession, more than just accepting Christ as your savior. Both kinds have common elements, as described by Richard G Scott:

  • Remorse: honestly recognize that what you did was wrong. Feel remorse for what you did, not just for getting caught.
  • Abandonment: have a permanent resolve to not repeat the mistake. And don’t do it again.
  • Confession: own up to what you have done. You can’t hide it.
  • Restitution: fix what you can. It’s not always possible to put things back in their original condition, but do what is possible.
  • Obedience in all things: you need to get everything straightened out. You can’t relent on one topic, but still harbor ill will on another topic.
  • Recognition of the Savior: this is a time when you should draw closer to Christ. Forgiveness comes because of Christ. Remember the price He paid for this opportunity, and at least feel gratitude for that. Mistakes separate us from God, and repentence and forgiveness is how we get back in His presence.

For what can I be forgiven of?

To quote from an article by Spencer W Kimball:

A young woman approached me in a city far from my home and came under some pressure from her husband. She admitted to me that she had committed adultery. She was a bit hard and unyielding, and finally said: “I know what I have done. I have read the scriptures, and I know the consequences. I know that I am damned and can never be forgiven, and therefore why should I try now to repent?”

My reply to her was: “My dear sister, you do not know the scriptures. You do not know the power of God nor his goodness. You can be forgiven for this terrible sin, but it will take much sincere repentance to accomplish it.”

Then I quoted to her the cry of her Lord:

“Can a woman forget her suckling child, that she should not have compassion on the son of her womb? yea, they may forget, yet will I not forget thee.” (Isa. 49:15.)

I reminded her of the Lord’s words in our dispensation to the effect that whoever repents and obeys God’s commandments will be forgiven. (See D&C 1:32.) My visitor looked bewildered but seemed to be yearning as though she wanted to believe it. I continued: “For all but the unpardonable sin forgiveness eventually will come to that transgressor who repents sorely enough, long enough, sincerely enough.”

She remonstrated again, though she was beginning to yield. She wanted so much to believe it. She said she had known all her life that adultery was unforgivable. And I turned again to the scriptures and read to her the oft-repeated statement of Jesus:

“All manner of sin and blasphemy shall be forgiven unto men: but the blasphemy against the Holy Ghost shall not be forgiven unto men.

“And whosoever speaketh a word against the Son of man, it shall be forgiven him: but whosoever speaketh against the Holy Ghost, it shall not be forgiven him, neither in this world, neither in the world to come.” (Matt. 12:31–32.)

She had forgotten that scripture. Her eyes lighted up. She reacted joyously to it, and asked, “Is that really true? Can I really be forgiven?”

Realizing that hope is the first requirement, I continued by reading many scriptures to her, to build up the hope that was now awakened within her.

How great the joy to feel and know that God will forgive sinners! Jesus declared in his Sermon on the Mount: “Your heavenly Father will also forgive you.” (Matt. 6:14.) This is on certain conditions, of course.

In modern revelation the Lord has said to his prophet: “Behold, he who has repented of his sins, the same is forgiven, and I, the Lord, remember them no more.” (D&C 58:42.) Our Lord gave the same word through the prophet Jeremiah: “For I will forgive their iniquity, and I will remember their sin no more.” (Jer. 31:34.) How gracious is the Lord!

On the occasion I am recalling, this woman, who was basically good, straightened up and looked me in the eye, and in her voice was a new power and resoluteness as she said: “Thank you, thank you! I believe you. I shall really repent and wash my filthy garments in the blood of the Lamb and obtain that forgiveness.”

Not long ago, she returned to my office a new person bright of eye, light of step, full of hope as she declared to me that, since that memorable day when hope had seen a star and had clung to it, she had never reverted to her sin nor any approaches to it.

It is important in all of this to forgive yourself. It is all too common that we seek forgiveness from the Lord, from the person we hurt, but fail to forgive ourselves. We should not forget the mistake that we made, so we can learn from it and not repeat the mistake, but we should not continue to punish ourselves after we have repented. To fail to forgive yourself is to mock the atonement of Christ. If Christ can atone for you because He loves you, and He can forgive your mistakes, you can forgive yourself.

So looking at that plaque on the shelf each week motivated me to do something. One night lying in bed I struggled to go to sleep, I was feeling compelled by a strong force to make it right, and do it now. So I resolved the next day to follow through. A search on the net turned up the person’s phone number in their new city. It had been several years, but I wanted a clear conscience. I picked up the phone and called. The person answered. Then a remarkable thing happened. He laughed when I announced my name. He said, “Just last night my wife and I were talking and somehow the conversation turned to you, the guy who lived down the street. I had forgotten your name, but my wife remembered it. Had you called before yesterday, I wouldn’t remember you.” Then I sheepishly started into my apology for what happened those years ago. He said, “I honestly don’t know what you are talking about. I don’t remember anything you offended me about.”

Giving Forgiveness

Not only will we find ourselves on the giving end of offense, but we will also be on the receiving side too. It is so easy to take that hurt that was inflicted upon you, keep it bottled up inside, constantly stirring and churning, until it ripens into bitterness and hatred and malice. It can be an acid that eats away at your character. It is playing out the victim to the nth degree. It almost seems human nature to do so. But what good does that do? It only hurts yourself. It doesn’t right the wrong. I have seen people who have been offended withdraw from the very things that they need the most because that other person is there, the one who hurt them once. Doing so serves only to punish yourself, not the other person. It doesn’t matter if that other person hurt you with intention or not.

The remarkable thing about receiving offense is that you will receive as much as you choose. This is a bold statement but I believe it to be true. You have control about how much offense you will receive. You can play as the fatally wounded animal, or it can be water rolling off a duck’s back. Sometimes from my children I hear about how a sibling did something wrong and “he/she made me mad”. I stop them right there and make a correction. No, they did not make you mad, you chose to be mad. You can choose how you react to any situation. If you find yourself mad, it is because you chose it. I’m not saying it is easy, it is difficult to control those feelings, but this is a true concept.

I guarantee that there will be frequent opportunities to receive offense. What you do in each of those opportunities is your choice.

At our core each of us wants peace. It is only through receiving and giving forgiveness that we can receive that peace.

The atonement is infinite. You can be forgiven and you can forgive. It isn’t easy, but it definitely is possible. If there is something you need to do, do it now.