In March of this year, I wrote a blog post entitled, “I Have Made Fire – Shortcut with Shell Script to Mount a NAS Share”. I was so proud of my little self because I’d written my first shell script to solve a real problem and I’d used a Shortcut with it just to prove that I could, not because it was necessary.
In May I wrote another article entitled “How Did I Break Keychain Access for My Local Server?” in which I explained that the fragility of Shortcuts made my solution entirely unstable.
In that second article, I explained how I tried to fix my solution and ended up somehow breaking my iCloud Keychain so that I couldn’t mount servers without pushing a button that said “Connect”. In a few thousand words I explained that I found a “defaults write” command that stopped the dreaded Connect button popup. I thought I was on the road to getting my original Hazel script to work.
All of that is still true, but the story isn’t over, because I still had to rewrite the script to include the functionality of the fragile Shortcut. For more than a month now I’ve been spending hours and hours trying to get this to work. I’ve asked the community in Slack and @lbutler jumped in to help. He gave me an idea that ended up being pivotal to the solution. The script goes inside Hazel, and the developer, Paul Kim has also been very helpful.
As I continued to run into trouble, I even tried Rubber Ducking the problem. This is a technique where you try to explain the problem to an inanimate object and the very act of trying to explain the problem helps you find the solution. Kevin Allder sent me a set of rubber ducks and I keep one on top of my monitor for just this purpose. I even named my rubber duck Kevin. I explained the problem in detail to Kevin (the duck) but he wasn’t much help.
Finally, I convinced listener and real-life friend Ed Tobias to rubber duck the problem with me in a video session. He’s a super nerd and does a lot of coding but he often tells me he’s not a “real” programmer because he does more hackery to get things done. I think that is a real programmer! In any case, I knew that Ed would be the perfect person to work through the problem with, and hopefully I’d figure it out.
I promise I won’t actually talk through any code at all, and still walk you through why this was such a sticky problem. It’s really a story of the inadequacies of the tools we have, and the constraints Apple has caused in the name of security. In order to explain, let me remind you of the problem I’m trying to solve.
The Problem to be Solved
I use the awesome app Hazel from noodlesoft.com to monitor my podcasting folders on my Mac. When the enclosed files are older than 2 weeks, Hazel whisks the files off of my internal drive and moves them to a share on my Synology. This is a critical automation for me because I create around 4GB of audio files every single week. If I don’t have Hazel cleaning up these files, I could easily run out of disk space and be in real trouble.
My Hazel rule works great … as long as I’m on my home network. The minute I leave my house, Hazel complains that it can’t find my Synology so it can’t clean up the folder. I’m barraged with notifications because Hazel so desperately wants to do her job. Hazel does have an off switch, so for years, I’ve been turning her off when I’m away from home as soon as I start getting notifications that she can’t find my server. That’s annoying because that means I also have to remember to turn her back on when I get home.
The other problem with turning Hazel off is she also watches some other folders that don’t require me to be on my home network, so I’d like those rules to keep working. For example, she watches my Delete Me folder and throws away files when they pass a certain age. All those rules stop running if I turn Hazel off. Instead of turning all of Hazel off, I could also pause just the rules that talk to my server, but that means a lot of manual intervention on my part, defeating the purpose of automation, and of course, all of those servers have to be unpaused upon my return to my home network.
I wanted to do one simple thing. I wanted to tell Hazel that if I’m on my home network, mount the server, and move the files. If I’m not on my home network, don’t even try to mount the server and just fail silently.
Conditional Section for Hazel
The first piece of the puzzle was how to tell Hazel whether I’m on my home network. That was pretty easy. I figured out how to send one single ping to the IP of my Synology. If the ping returns, then I’m on my home network. If there is no reply to the ping, then I must be out galavanting around on other networks.
Hazel has two distinct parts. The first part is the conditional section. My conditional statement for my podcasting files is, “If a file is older than two weeks”. On @lbutler’s advice, I put my ping to test if I’m on the network up in that conditional section. In Hazel, you get to choose whether any or all rules have been met, so I set it to all conditions. The conditional section then says, “If Hazel can ping the Synology AND has found a file that is older than two weeks”, then it is allowed to proceed to the next section where actions will take place.
I was able to determine that the conditional section was working properly by using a built-in function of Hazel, even though I hadn’t yet set up the action section. You can choose a file and Hazel will tell you whether any or all of the rules have been matched, including my script to ping the server. I was able to flip back and forth between my main network and my guest network where the Synology was unavailable, and Hazel behaved as expected.
Action Section of Hazel
The second half of Hazel is where you tell her what to action to take. My goal for a script in the action section was pretty simple. We know that we’re on the home network because we passed both of the conditional rules tests.
The action statement should have been the easy part. I only needed Hazel to mount the share, and then move the old files to the server. This is where I ran into the difficulty with Apple’s security constraints. While I can use ⌘-K to mount my server from the GUI, if I use the mount command from the Terminal or a shell script, I get a popup showing my credentials, but it requires me to push the button, even though my credentials are stored in iCloud Keychain. Obviously, if I’m trying to have a script run to do this, I can’t have it pushing a GUI button.
This is different from the problem I described in my post entitled, “How Did I Break Keychain Access for My Local Server?“. In that case, I was just getting a “do you really want to connect?” button but it wasn’t asking about my login credentials.
I chatted with @lbutlr about the fact that I couldn’t mount the server without this GUI interaction, and he suggested a different approach. He built a very simple Automator script to mount his server, and he thought it might work for me. Automator is a drag-and-drop environment where you can also add scripts so it sounded like a good path
Following @lbutlr’s lead, I dragged in the Action called Get Specified Servers, where I typed in the SMB mount command for my server and it’s share: smb://192.168.4.95/synodo-al-backup. The second Action to drag in is simply called Connect to Servers.
As @lbutlr promised, I can run that Automator script and my server share mounts with no popup from the GUI asking if I really want to connect. I was so happy! But my joy was short-lived.
These two simple actions work perfectly unless the server share is not already mounted. If the server share is already mounted, Automator throws an error. Well, that’s just silly.
I would need to tell Automator to only try to mount the server if it’s not already mounted. That didn’t sound too hard. (Queue the foreboding music.)
Automator Isn’t Logical
This is when I discovered a huge limitation of Automator: it has no logic capabilities whatsoever. It has no concept of “if this then that.”
I should be able to work around that. I figured I just write a shell script to put between the Actions “Get Specified Server” and “Connect to Server. In my script I’ll put the logic:
If the server is not mounted
then mount the share
If the server is already mounted
do nothing
fi
That’s a pretty simple set of instructions, but it didn’t work. It turns out that you simply have to give the second Action, “Connect to Server” a valid server address. Since – no matter what – that last “Connect to Server” action will be executed, if you don’t pass it an address to which it can connect, it throws an error.
Chasing My Tail
At this point, I felt like I was chasing my own tail. For weeks I would try to get this to work from the shell script but I’d get that darn GUI popup, so I’d switch back to using Automator but then I’d run into the already-mounted server error. I kept thinking if I could write a better script, I could get out of this.
At one point, I took a drastic step and checked the Hazel log files in the Console App. Console is the place where madness thrives. I’ve looked at log files so many times, and not once did I learn a darn thing. But Paul Kim, developer of Hazel, actually writes out log files in a surprisingly human-friendly way.
According to the Hazel logs, with my Automator script embedded inside Hazel, the “Connect to Server” Action was very confused. Rather than being passed the server IP and share name, somehow it was getting passed the name of the file that Hazel had found was older than two weeks.
Developer Paul is super helpful and super smart so I emailed him about this. He explained that Hazel always passes the file name from the conditional section to the action section. Since my first action is Automator, it is the lucky recipient. Paul went on to explain that Automator has an option on each action to ignore any input it receives.
This was super important information to have, and nothing I was doing was going to work if I hadn’t known to implement this solution, but it didn’t change the tail-chasing I was doing between shell scripts and Automator to reach my very simple goal.
Before I go on, I want to tell you a funny little thing. When I was reading Paul’s console logs, I wrote in our Programming By Stealth channel in Slack, “Proof that I’ve descended into this shared madness called programming. I just successfully interpreted an application’s log files. I AM a developer!” Scott Willsey made me laugh out loud when he responded with “Clearly the app’s developer forgot their lessons in log file obfuscation.”
Rubber Ducking Works
After chasing my tail for weeks, it was time for my rubber ducking session with Ed Tobias. I wanted to keep the conversation at a high level, so to avoid digging into syntax, I started a screenshare and simply typed out logic statements such as, “If share is not mounted, then mount share, else do nothing.” Once he had the big picture of what I was trying to do, I demonstrated the two problems: the shell script causing the GUI popup, and the Automator script falling over in a heap if the server was already mounted.
As we noodled the various issues with solutions I had tried, it suddenly hit me – there WAS a solution.
If Automator can mount the server share as long as it’s unmounted, maybe the first step in my script should be to unmount the share. When I told Ed my idea, I could not stop laughing at how my solution was stupid and yet also brilliant.
When Ed and I were finished reworking the solution, the entire function of the script was to find out if the share was mounted, and if so unmount it, then pass along the SMB connection information to the Automator action to mount the share.
We tested Hazel with my now very simple Automator/shell script under three scenarios:
- When I was on my guest network – as expected it did nothing at all and I got no notifications
- On my real network with the server share not mounted – it mounted the share and moved the old file as expected
- On my real network with the server share mounted – the share briefly disappeared, reappeared and the old file moved as expected
With one last giggle at the solution, we ended our screen share and I declared victory.
What Can Possibly Go Wrong?
I had my first chance to real-world test of whether Hazel would fail silently when off of my home network over the Father’s Day weekend. We surprised Steve’s dad by appearing on his doorstep for Father’s day (they live 4 hours away), and we stayed in a hotel using the free WiFi. My VPN from PIA kicked in, and over the weekend, Hazel had no complaints about not finding my server. I was very happy.
But then as I was writing the very last line of this article, I had a thought. Hazel’s job is to run silently in the background keeping an eye on your folders and taking action according to the rules you set for it. What if I’m at home working on a spreadsheet that’s on my Synology, and Hazel finds a file that she needs to move? She’s going to unmount the share and remount it just as instructed. That would cause an interrupt and possibly corrupt my file.
Automounter to the Rescue
Sadly, I had to abandon my brilliant solution of unmounting and remounting my share before doing any file moves with Hazel. Earlier while I was searching for a solution to this conundrum, I found a freemium app in the Mac App Store called Automounter.
I think I do have a solution though. I’m going to keep the rule that tells Hazel if I’m not on my network, then don’t do anything, but I’m not going to even try to mount (or unmount) the shares. Instead I’m going to run the $10 Automounter App from PixelEyes and available in the Mac App Store. PixelEyes is a New Zealand company so they must be good.
Automounter’s entire reason for being is to keep server shares mounted. If one disconnects for some reason it just brings it right back. I hadn’t embraced Automounter as my solution because the idea of having all these server shares from my Drobo and Synology mounted all the time cluttering up my desktop made me crazy.
But I gave Automator a try anyway, and it has a very slick feature: it can hide mounted shares so you don’t have to look at them on your desktop. They still show in the Finder sidebar so there’s no problem accessing them, but they’re not in your way. I can choose to not worry my pretty little head about them, and yet they’ll already be mounted when I need them.
I configured Automounter to mount my server shares and it worked just as it promised. I paid $10 for the Pro Settings which allows you to define custom mount points, such as /Volumes, and you can also have applications or files launch when the share is mounted.
Now the question was how would it react when I went on travel? Would it start sending me notifications that it couldn’t find the servers, just like Hazel?
Nope! I checked Automounter from the hotel WiFi, and in the listing of the shares I had instructed it to keep mounted, it simply said “SMB not available”. It also says “Rules not Met” at the top.
Bottom Line
While it still bothered me that I couldn’t crack the code (literally) on this problem, I discovered Automounter and it essentially solved the symptoms so my Hazel rules will function and not nag me when I’m away from my home network.
But then I got an email from Ed saying he had another idea for a script…
And then I found in Paul’s excellent documentation that Hazel has a built-in Connect to Server feature that I forgot about…
🎶 This is the song that never ends… 🎶
Hi! Reading through this to get some ideas about Hazel, and thought I might toss out a couple of ideas.
First, using “ping” to look for the Synology NAS in determining whether you are on your home network is pretty clever. I think we can make this a bit better, though.
1. The standard UNIX “ping” command, including the one on my Mac (tested just now) returns a “0” (zero) exit code if the ping you used succeeded (testing against my Synology). This is in conformance with standard Unix Philosophy, where a command returns zero on success and non-zero on failure. (And maybe any of several different non-zero, depending on what sort of failure. For “ping”, execute the command “man ping” and look for the “EXIT STATUS” section, near the bottom.) And when I tried it with an IP address that does not match a device on my network, I got an exit code of “2” (conforming to the EXIT STATUS section of the man page). And, in order for your “if” statement to work, it must be doing something very similar for you. Because if the ping did not return truth (0), the “else” clause would be executed. And so, I think this can be simplified. Basically, you just want to return the exit code that is emitted by your ping command, whether zero or not. (Unless Hazel is looking for something really weird in terms of exit code return.) Now in a traditional shell script, the script’s exit code is the exit code of the last command executed in the script. So, if the Hazel scripting works that way, you could have just the ping command all by itself. And if it doesn’t work that way, the Hazel folks should be told, suggesting that it would be a superior way of doing things. So, if the Hazel script does not simply return the exit code of the last command run, then you could have your ping command, followed by the line, “return $?”. In most modern shells, including zsh, “$?” is replaced by the exit code of the previous command.
2. My Synology NAS has a name. To give yours a name or check what its name is: Sign into your NAS as the administrator. Open “Control Panel”. Select “Network” from the list of sections on the left. In the default “General” tab, near the top, there is a text box for “Server name”. In there, I have “HeibySynology”. And, because magic, that allows me to basically ignore the IP address that my router has assigned it, and allow me to refer to it, not as “192.168.nn.nn”, but rather as “heibysynology.local”.
Now that we have simplified things a bit, I’m wondering whether they need to be more complex, maybe.
I am thinking that it would be useful to configure my Synology to provide me with VPN service, so that when I am remote and using some random Internet Wi-Fi hotspot, all of my traffic is encrypted and protected from the random hotspot provider, and instead is known only to my normal ISP at home. (Discussions on whether it is a good idea to trust my ISP or not are not entirely germane here, since I do trust (sorta) my ISP while I’m sitting at home.)
And, I think that if I have Synology providing my VPN, it would be handy to be able to fully access my NAS, printer, etc. at my home from wherever I am, making my remote Mac think that all of those things are local to it.
But if I do that, then the ping will succeed, since my remote Mac, via the VPN, is logically part of my home network. And is that a good thing? I’m not sure I know. I suppose that it depends. Maybe I’m fine with copying files to my NAS while I am remote and connected to my home via the VPN. Maybe I’m not, because maybe I’m not connected to a coffee shop’s Wi-Fi, but am instead connected to a metered wireless hotspot (or phone’s hotspot), so maybe want to keep my data use to a minimum.
Anyway, throwing the VPN into the mix probably means that once I have that set up, I need to have some way of detecting whether I am physically present at home or whether I am logically present at home via my VPN. And with that, maybe I can use different Hazel rules, dealing with small files whenever, but large files only when actually home, unless they are in a folder I consider high priority to copy across, whatever the data charges. Yup, complicated.