Velocity Conference Amsterdam

They are everywhere. Nearly all the talks mentioned them in some fashion. It seems these are going to become more and more important over the coming years. While at their most basic they let you scale your website to fit on many device types they are starting to add a lot more to apps in general. There is support for notifications and adding the "app" to the desktop or application menus in the browsers. This means that you can create what feels like a native application using web technologies. We are already seeing people ship applications that are nothing more than a thin wrapper around a web rendering engine. You should be looking at these now. Even if you are not using all the features that they suggest you can already be taking advantage of the boost in performance you get and helping future proof you a bit.

There where a few talks about http/2 the new version of the standard that drives the web. http/2 is out there and supported by all modern browsers now. Talks varied from the W3C proposing new additions to improve performance to people writing servers and studying how browsers interact with http/2. While some of the newer and deeper technical details where overkill for general use it's still good to know how these things work under the hood. Like how a web browser will load as far as the end of the head section in a web page and then build a download map for what to do next. It might switch to download the images and stylesheets first then the rest of the page. You page, site and server can help the browser with these decisions by providing maps and hints. Push is a part of http/2. The server can push some things that it thinks the browser might need next. This leads to servers pushing things that the browser already has cached. So now there is a cache digest that the client can send with the first request so the server knows what the client has cached. While browser support is patchy there is a cacher-digest.js script to fall back on. The upcoming QUIC protocol based on UDP seems interesting as it cuts out a lot of the round trip time (RTT) for a packet to get to a client. These can all add up so by doing more things asynchronously the overall speed can be improved. There are lots of changes in http/2 some of which are easy to get the benefit from and others will only become obvious as app frameworks adapt to these new paradigms. In the meant time it looks like it's about time to turn it on.

There where quite a lot of talks about the soft side of the craft. One of the keynotes was themed on how a diverse team is a more productive team and hiring in your own image ends up creating a team that is fixed in its ways. Other talks where about how teams have scaled, switching to smaller mixed teams with cross group knowledge sharing. I think my favourite one though came from a member of the wellness team ( What your enterprise might call HR) for the German company who talked about their review process and how that works when you don't have any managers. They essentially get people to volunteer to be reviewers, select the good ones, and then everybody gets to pick who does their review. The opening talk, Word Done vs Work as Imagined, touched on many things but a couple that stood out where about setting metrics and automated alerts vs an human skill and experience. On the first point they used the fact that too many alarms went off on a flight and the pilots had to ignore them and fly using experience. This maybe true to some extent in this instance but it also might point to bad alerting interface. I would recommend that you read The Checklist Manifesto by Atul Gawande at some point though for an interesting looks at how sometimes human instincts need some backup. Next up was the measurement, reward and punishment of workers using metrics a firm favourite of the UK Government. I think it still bears repeating that you get out only what the metrics are requiring of people so be careful. If you set a 4 hour minimum to be seen in A+E then staff, even good ones like doctors and nurses, will bend and break all the other rules to achieve that. Systems are invented to meet the target even if that is a worse way of doing things.

Containers have moved on and are no longer a big topic at conferences I think. I don't know where they are on the hype curve but it seems that the assumption with a lot of the talks was that you where already running them. Topics like Immutable Infrastructure and even Serverless did get a mention now and again. It feels like the technology stack has shifted slightly again. People are concerned with Docker Swarm or Kubernetes for managing applications rather than managing servers specifically. It seems assumed that you will be running on some sort of "cloud" where groups of "servers" come and go. Now we are looking at how to make systems tolerate failure rather than be 100% reliable. Gone are the Dual power supplies and redundant networking being replaced with autosizing clouds and redundant cloud suppliers. Fun times.

Metrics are key to any sort of modern development. They give insight into what is working and what is not. They let you test changes and make sure they are not breaking things in subtle ways. Gathering and processing metrics is hard. We can very simply monitor systems and times but figuring out what is important or what an issue looks like is more tricky. It does not seem that long ago that the #monitoringsucks hashtag was all the rage, closely followed by the slightly more positive #monitinglove one. At the time Nagios and Cacti where the tool and we struggled to make use of them where we could. Now it feels we are spoilt for choice. There seemed to be two main themes to the talks those talking about how to gather the statistics and those dealing with interpreting them. On the gathering side probably the most interesting for me was the description of the close integration of Kubernetes and Prometheus. These two tools seem to be the very hot at the moment and are starting to change the way we think about our infrastructure. Completing the move from snowflake servers that we nurture and run applications on to 'pods' that house applications that we run on clusters. With this shift in complexity we need new tools to manage this. Kubernetes handles the running and scaling of the applications themselves while Prometheus gives us the eyes into the system. The other side of metrics is interpreting them. I think my favourite talk here was 'How the users see the data'. It gave a whistle stop tour of how to display metrics. A lot of it based on the work of William S Cleveland and his Graphical Perception paper Showing how people read graphs and giving advice on how to display data to get the most meaning. Some examples included "Stacked anything is nearly always a mistake as are pie charts. Use two charts or only show the important data". Or that there is a scale on which people are better are interpreting data that goes from Position on a common scale to shading. So we are much better at recognising values and difference on a scale than we are say angles or direction. There is a lot more to it than that but I feel I learned a lot there. Anomaly detection was also a theme that cropped up, people have differing opinions on what works and what does not but when you are starting to gather large groups of statics. I attended the tutorial on the last day about using some basic AI and stats in python to spot anomalies in your data even if its not regular. While it was interesting it did feel like it needed a lot of manual work to get the best results so you might need to focus on just a few important metrics.

The concept of Continuous Delivery is well established in the DevOps world. Keeping the software always deploy-able at all times. Security on the other hand is much more of a mixed bag. The tag DevOpsSecBiz was proposed and while the name will probably not take off it does point to a different approach. First things first you have to get a commitment from the project sponsor or manager that security is important. Then you can start to apply all the techniques that you learned from DevOps to add in security. Sit your teams together, all working towards the same goal including security. Don't make it a "security team" problem that you tack on at the end. Start the project with threat modelling, get your developers to buy into this. Then automate the tests, add them to the infrastructure. Start adding tests to your code for security, unit tests that reflect your goals. Start running checkers for bad code smells and practices in the builds. Add in automatic scanners in test and production. Code reviews, OWASP ASVS is a good starting point. I think one of the final points of the talks was interesting a focus on the security team not being off somewhere "handling security" and probing and testing but being part of the teams improving process and culture.

The final day was tutorials. This was a mixed bag. I chose to do a mixture of topics so I would get a taster of each. It was great to learn about optimising images for the web lots of good things in that one. Getting to explore the basic data science allowed me to think about some of the talks from the previous couple of days. The first tutorial was a programming one using eBPF, which is a low level kernel task that is really powerful. I could have spent most of the day playing with that but it seems to be at odds with the worlds of containers and disposable infrastructure. It does highlight how the split is forming, there seem to be now companies that provide large infrastructure and have the need to dig that deep and a move for most companies to consume those container or even functions hosting services. I do wonder how long my role as a generic systems administrator will exist. Where should I aim for next?

The conference felt very large and quite anonymous. I managed to speak to quite a few people over lunch and I think things like the birds of a feather (BOF) tables at lunch helped open people up a bit. As ever with techy conferences though I think this can be an awkward time for a lot of people. It felt a shame that there was not something on after the event. People stood around chatting to their peers or the vendors for a while then went off. I have been to other conferences where that time seems better spent in a group. ( Having said that by day three I was shattered! ) The packed schedule gave little time to thinking, which may seem weird but as I read through the notes I managed to scratch out in between sessions it seems I could have done with some more time to digest some of this. Not sure there is a sensible solution to that one though.