• Over the years working with developers on all major desktop platforms, I’ve observed an interesting pattern. Much of the web development tool ecosystem is built on and around open-source projects, which traditionally favour Linux. So when a .NET developer, running Windows, jumps into working with Node / Golang / Python, they are immediately disadvantaged by tooling barriers and lack of community content and support.

    Windows creates constant friction for OSS-based web devs to work fluidly.

    This does not affect macOS, thanks to its Unix underpinnings. But many developers do prefer to use Windows as their primary desktop OS - be it because of other work they are doing in Microsoft-backed environments, or just because gaming. Whatever the preferred OS, the work needs to happen. But it does take extra effort on Windows.

    We, as DevOps practitioners, will do whatever is needed to build as good of a web dev experience possible in the Microsoft ecosystem. But I feel this experience will always lag behind that of working on Linux or macOS, just due to the sheer size of the ecosystem shared by, and targeting, the other two dominant platforms.

    An example:

    • Everyoneâ„¢ either already uses, or wants to use, Docker.
    • Also, many devs use VirtualBox, because this free virtualization platform is what a lot of the tooling and community knowledge are centered around.
    • But, on Windows, Docker needs Hyper-V.
    • But! Hyper-V is incompatible with Virtualbox.
    • But!! Virtualbox has the broadest ecosystem of getting shit done in a fun automated way (Vagrant, basically).
    • But!!! Lisa needs braces Docker needs Hyper-V…

    /shrug.

    There are ways of making it better.

    • Use docker-toolbox. Yes, this is a legacy solution, but it will allow you to get around the Hyper-V requirement, and use Vagrant and Virtualbox for other needs.
    • Write tooling around it that is cross-platform. Use cross-platform task runners. We’ve had good success with go-task
    • Write toolchains to wrap your workflows into, and run them in Docker. This gives the added benefit of portability of your development environment across machines and even platforms.

    There exist other ways to ease the path of a Windows web dev, which I will explore in future posts. Holler @boomstik on twitter with your thoughts, comments and ideas.

  • Wed, Jan 3, 2018

    “Oooh, Azure Portal, how I’ve missed you!” - said no one ever.

  • jq is a swiss army knife for working with JSON. It is especially handy for piping output of CLI tools, such as curling JSON APIs, or aws and az CLIs.

    I wanted to get a nice list of public IP addresses of my EC2 instances, together with instance names. I could have used boto for this, but the combo of AWS CLI and jq turned to be a simple and effective one-liner (split for better wrapping).

    aws ec2 describe-instances |
    jq '.Reservations[].Instances[] | {(.Tags[] | select (.Key == "Name") | .Value): .PublicIpAddress}' |
    jq -s add
    

    produces:

    {
      "foo": "54.131.121.177",
      "bar": "52.75.8.58",
      "baz": "34.228.156.28"
    }
    
  • Azure functions can look at blob storage and react to things.

    But actually not really all that well.

    Excerpt from the Documentation:

    When you’re using a blob trigger on a Consumption plan, there can be up to a 10-minute delay in processing new blobs after a function app has gone idle. After the function app is running, blobs are processed immediately. To avoid this initial delay, consider one of the following options:

    Use an App Service plan with Always On enabled.

    Use another mechanism to trigger the blob processing, such as a queue message that contains the blob name. For an example, see Queue trigger with blob input binding.

    Let’s deconstruct this a bit.

    The important parts are the "Consumption Plan" vs "App Service", and how those relate to the Always On mode.

    See, Azure Functions have two methods of operation (“plans”). The “Consumption” plan executes the function only when triggered. So if nothing is calling it, the function will go to sleep. A Function runs ephemerally and you need not think of its underlying resources whatsoever, aside from paying per invocation.

    The App Service plan, on the other hand, launches a VM that will host your functions, and that VM remains running. You don’t need to directly manage it (nor can you), but you are being charged for all the minutes it’s humming away. Also, unlike the Consumption plan, you need to manage autoscaling yourself.

    Only on the App Service plan you are given the option to enable “Always On”, which will prevent your function apps from going to sleep.

    So in contrast to the probably familiar pattern of AWS Lambda being triggered by a change in S3 bucket, the Azure Blob storage doesn’t immediately trigger your function on change in blob storage, unless the function is already running. Otherwise, you are waiting for the scheduled wake-up window (feel free to correct me on Twitter if I am misunderstanding something). I personally find this behaviour to be super confusing, and inferior to what the rest of the cloud has come to to expect of the “serverless” patterns.

  • Good morning. Today we will take the terms “domains”, “fault”, and “update”, and make it sound more sophisticateder because competitive advantage.
    - Azure marketing people, probably

    I mean, it’s good they have thought of this. It’s even on the exam. But really, as the user of Azure, I don’t need to care about how they power their racks and in what order they are restarted. I care about stability of my VMs, but it’s ok to leave the mechanics of fault-tolerance to be a black box. For the most part, it would suffice for me to know that if I launch a group of 3 machines, I’ll have almost 3 machines running most of the time. I don’t have any control over this anyway, so those “domains” are trivia and implementation details.

    That aside, Microsoft’s general aversion to visual presentation of data rears its ugly head here once again. They could have designed the UX around this as a nice grid, with current status of each slot in the fault/update domain, etc. Could’ve even put this next to each VM. But no. Everything must look like a spreadsheet.

    The important takeaway of the entire feature: You should, for best availability vs cost effectiveness, try to horizontally scale your VMs in sets of 5: N % 5 == 0. That’s how many update domains exist. N < 5 - and you’re not utilizing the full fault-tolerance potential. 5 < N < 10 - and you are overprovisioning some of those update domains.

  • Some of these tech vendors need to grow a pair.

    I found an email of this nature in my inbox today:

    Dearest Eugene,

    Our sincerest apologies… (sob)… for previously mistakenly sending you this email about some industry event that we’re hosting. We realize you have not registered, and yet the message was dispatched to you. HOW COULD WE. For the inconvenience that we have caused you, and all the confusion - we are so, so sorry!.. We truly understand the rollercoaster of feels that you’re experiencing at this very moment! Please believe us, this was a mistake. We want - nay, need! - to do better. You DESERVE better. We admit, this was a human error. We made a mistakeWhat a mistake… And, indeed, we do so sincerely hope you still choose to remain on our mailing list. Please, please let’s stay friends! However, if you choose not to (ohnoes!!)…. We will be sad. So very sad, yet understanding of you clicking this unsubscribe link. ( pleeeezdontgoooooooooooo )

    (Of course I’m exaggerating, but only slightly).

    seriously, what IS this shit.

    You’d think this was an apology for - i don’t know - accusing me of eating live kittens on national TV, or something of similar magnitude. But alas, all it was was a mere mis-addressed email message.

    Now, some of the content on their mailing list is interesting. But this type of submissive servility is actually a turn-off of unsubscribeable magnitude.

    Dear tech copywriters of the Web 2.x. We are not children, kk? Most of us don’t need a safe space to cry over your accidental emails. KTHXBAI.

    Though on second thought, if you really feel the need to apologise - nothing short of a phone call shall suffice. Ladies and gentlemen - start your Asterisks!

  • So you get an idea, and it’s an amazing one. You’re inspired to start working on it right away…. But what if someone has already made this? Does this mean your idea is dead?

    And so you rush to the interwebs, and prepare to search…. But wait!

    Until you’ve made the search to determine originality of your idea, it is simultaneously both dead and alive.

    Meow.

  • Technical debt doesn’t just accumulate. It incurs interest, in form of features and dependencies that will then need to be refactored when this debt is repaid.

    Is it ever possible to repay it completely, though?

    Defaulting on technical debt is when you scrap the whole thing and rewrite it from scratch. Thankfully, this does not lead to technical bankruptcy.

    Except maybe when your API is written in BASIC?

    10 PRINT "LOL"
    20 GOTO 10
    
  • Fri, Dec 16, 2016

    It took me 3 years to grok callbacks in JS. Not something I’m proud to admit.

  • Wed, Dec 14, 2016

    Propensity for dogmatic brand loyalty in a person is indicative of their critical thinking skills.

  • Recovering my better half’s system drive. Her OWC 480GB SSD was allowed to reach 100% capacity (only 1.5GB remain)… And all hell broke loose. It’s barely readable (takes 10 minutes just to mount on my laptop), and I can’t even delete anything (any attempt to modify the filesystem just returns an invalid argument).

    I’m currently rsync-ing all the things to a network volume, and will attempt to deal with this after the data is safe.

    That machine was long overdue for a refresh anyway.

    Lessons *:

    • When setting up an SSD, make sure to enable TRIM. Windows | macOS (≥10.10.4).
      • OWC seems to discourage the use of TRIM, citing “garbage collection”. I believe those are different things, but more research is needed.
    • Leave unpartitioned space on the SSD. The accepted guideline seems to be ≈10% of total capacity

    * Disclaimer: I take no responsibility whatsoever for any effects, including but not limited to loss of data, caused directly or indirectly by this blog post.

  • Today I was testing how a SaaS that we use for search handles failover. I clobbered our DNS to make our app use their secondary endpoint. The module in our app handled this very well, and switched over seamlessly.

    But then a thoughts crossed my mind: did this affect the Ops folks on the other end? Did someone see a blip on the charts, and ask the person next to them:

    hey dude, are you seeing weird traffic on node B13? dafuq is that?…

    Or… was it simply missed? unnoticed, lost in the shuffle - because let’s face it: ain’t nobody got time for this.

    I guess the former would be nice, because that would mean those packets of mine were just a little more special than the rest of the internet noise.

    Yet I do hope for the latter…

    Because Karma.

Hosting AWS Docker Microservices Tooling Automation