Continuing along the Elasticsearch Scaleset process; I’ve attempted the virtual machine extensions mentioned previously to little success. The expected files are created but are empty, and therefore don’t do the job. Probably some sort of permissions issue. I’ve moved on to trying to handle them with a startup script with somewhat better effect thus far. And in point of fact, a startup script is likely to be a better bet anyway, since (provided the startup script is delivered in the packer image) new scaleset members will start up with that script having run; it’s not clear that he VM extension would run for new scaleset members, and I strongly suspect it would not. I also discovered that the eval process I’m using for bash file-templating is … unuseful for some of the files - one bash-script-like file in particular has a number of env variable references in it like $ES_CLASSPATH or $ES_HOME which, when procesed through the eval process, end up empty and making the file invalid. I had to switch to using sed to do the replacement(s) I needed in that file. #

Of course all of this is till running into startup problems, maybe at least partly to do with not correctly handling (or not properly ignoring) failure modes? And the challenge of running it @reboot is that if it fails the system, doesn’t boot. I’m not 100% sure why this is; most of what I’m doing in there shouldn’t be relevant to system startup (aside from whether or not Elasticesearch can start, which is a separate issue and shouldn’t mean that the OS boot would fail). #

Deallocating a scaleset instance seems to take a much longer time than I would have expected, as well, and oddly, for some script failures a deallocation and start cycle seems to fix the issue. #

I’ve been looking at using an alert filter in Azure connected to a webhook to publish notifications (perhaps to Slack). Might be useful not only for knowing when a long-running operation is finished but also for tracking other events. Interestingly enough the Administrative event fiolter also published activity from the Azure CLI (and presumably from Azure-PowerShell). It’ll be interesting to see if it also catches terraform activity. #