Dear person who is not in the community of software developers but is, in one way or another, responsible for your organisation’s most important system running in a US cloud. This text aims to bring hope.
A few days ago, I wrote “Don’t panic, but it’s time to take action.” As I said there, I hear lots of people suddenly being worried about their core systems running in US clouds. They see a new situation, and they feel trapped. I said that there is hope, and I will try to share some orientation and tips for you on how to tackle this together with your best friends, your tech team.
I also expect colleagues in the community to chime in with more helpful information.
Note
Since I think time is very important today, I chose to publish this in a sketched format today. I will reiterate it and improve it. But… Time is very important today.
Target audience
Please note that this text is not for the community of software developers. It’s rather for you who is somehow responsible for the most important systems of your organisation. The systems that are custom developed to provide differentiation and competitive advantage. The systems that determine whether your organisation fails or succeeds. The same systems now happen to run in US clouds which for many is an unwanted situation considering the evolution of global politics.
Eight months ago I wrote something related (in Swedish) regarding the EU regulation “Digital Operation Resilience Act” (DORA). The companies being affected by that should have this under control since January 1, as I understand the regulation regarding the requirement of an exit strategy. This text is rather meant for all the other companies, not “only” those that are important for the financial stability of EU countries.
The text also assumes that you work in a European company and that your system has European users and/or manages information about Europeans. That said, quite a lot in the text is generic to any move of a custom-developed system though.
More people who are not the target audience
Of course, running standardised systems in US clouds might also be risky, but that’s a different story altogether. It also requires work to solve, but in that case, you are among a big group of other companies sharing the same challenges. I will not talk more about that in this text. I’m sure others who are much more knowledgeable about that than me will share advice.
@Community, please help out with solid advice on how to move from standard package X to Y.
Why now?
I think a lot of people have been considering for a long time to use European cloud providers rather than the US ones, to avoid transferring data about Europeans to US companies. The current agreement for allowing that is called EU-US Data Privacy Framework and has been at play for almost two years now. (That is after the two prior agreements had been considered invalid by the European Court of Justice). The current one seemed like a weak construction from the start, I think it’s just waiting to be invalidated. That said, I think the convenience of staying in US clouds has been considered by many to be stronger than the risks. After all, even if the paper construct was weak, there used to be quite good trust between the EU and the US.
In the last few days, the trust situation has changed (understatement of the day?) and the risk has skyrocketed. It will require work and therefore cost money to move, but the valuation of cost vs risk has changed. The legal situation is probably troublesome very shortly, and the moral situation since a long time ago. Add to that the risk of threats to close down access to the US clouds, unthinkable just days ago.
I think it’s reasonable to follow the law, to not expose information about users, to avoid vulnerabilities and keep dependencies under control. I’m sure you do too. If that means that you should do the move or not for your system, that’s for you to judge.
Common misunderstandings
Before we continue with the subject of this text, how to move from US clouds to EU ones, I just want to briefly clarify a common misunderstanding. It does not help that the US cloud you are using stores the data on European soil. What matters is the country of legal residence of the mother company. The reason: they could be required by US law to provide data to US authorities.
Another common misunderstanding is whether you are handling sensitive information or not. From a European law perspective, what is considered personal data is very broad. Take an IP address for a user of your system, for example.
With that out of the way, let’s dive in.
Cost
It’s of course hard to say much about the cost before doing the analysis which we will talk about shortly.
I was just thinking about a situation quite many years ago as a reflection to this discussion. We were doing technical due diligence for a prospective buyer of a big software company. We were very concerned because we came to the conclusion that it would cost a couple of millions of euros to fix the architecture and to get them in a reasonable shape for future development. We reported that in one of the daily meetings with all the different due diligence teams. The prospective buyer thanked us and said that we were done then because that was peanut money compared to the other real problems. :)
My point is that cost is always relative and a matter of context. To a techie, it might seem extremely expensive to do the move. To the business owner the cost might be almost negligible compared to the risk of staying.
Time for action, where to start?
Again, we are talking about your custom-developed system, unique for you to differentiate your business. Therefore, the situation is unique and context is king. That said, there are also similarities between the situations, and common approaches could be applied. I’m not talking about “best practices” here, those are scary to use in unique situations. I’m rather talking about some common and proven ideas‚ rules-of thumb if you will, that you can see how they fit your situation. When they are applicable, use them. When not, do something else.
To start from the beginning makes a lot of sense as usual. A generic approach would be to first do an analysis to understand the current situation then to come up with a vision, and then to understand the gap between now and the vision.
The generic approach fits well here. The second and third steps will be easier than in many other situations though, and that’s good.
1. Current situation
Regarding the current situation, first focus on finding out where you have tight dependencies to the specific cloud you're currently using. When those decisions were taken, that was probably for good reason (even though I suggest the default would be to aim for being cloud agnostic). Now those decisions have become problems that you will have to deal with. Common problems will be infrastructure dependencies such as:
- message queue
- database
- logging
Most systems use all of those. What we are looking for is dependencies to cloud-specific solutions that you won’t find in your target cloud. Please note that it’s much worse if such dependencies are spread out in many different places in the codebase. The difference can be enormous. From having the possibility of changing and testing the change in one place to having coupling to the database for example “everywhere”!
If you don’t find any dependencies to the specific cloud, part of the analysis should be to make a test move and thereby get a list of less obvious challenges.
2. Vision
Most often, architecture modernisations are driven by the possibility of a new or bigger revenue stream. In the case we are talking about today, it’s rather driven by the will to reduce risk. Maybe not as inspiring, but just as important. And actually quite a bit easier. The “vision” in this case is probably something very straightforward, like “host our system in a European cloud”.
A decision that is probably less straightforward is of course the choice of a new runtime environment. I will not delve into that here, but only say that even if it might be a hard decision, I don’t think of this as a dramatic one. You should of course choose as wisely as possible, but you should also use this transformation to create flexibility for moving again if the need arises. After this first move, you will actually be in a much better position in many ways!
@Community, can you please help out with advice on how to choose a new runtime environment?
3. Gap
Again, this step is quite easy compared to a modernisation journey, where you’re aiming for something that hasn’t been done or even thought of before. In the case of moving to another cloud, we can actually know what we need to fix so the system can be moved to the new runtime environment.
A checklist to evaluate for possible initial actions
I don’t suggest that you create the perfect plan with steps for all the actions in your gap analysis. Rather that you decide on the most important first and minimal action and execute it. Then you re-evaluate and decide on the next minimal action. Getting as quick feedback as possible and keeping manoeuverability is key.
Even though I’ve said that the vision and the gap might not be as hard to come up with, you are still operating in a dynamic environment in many ways. Just to give a simple example, when executing the first action, it might affect the overall situation so that what you thought would be the second action isn’t needed at all or is way less important now.
Another reason is that the business will not shut down during your actions. Since your system is crucial for your company’s competitive advantage, it’s more than likely that changes will be needed occasionally. Those changes will also affect the landscape.
It’s also the case that you can’t avoid learning a lot as you’re progressing through your actions. That learning is super important for making the best progress possible. You will never again know as little as you did before you started.
Therefore, take advantage of the fact that the situation and understanding evolve and re-evaluate what should be done next when an action is taken.
What will be your first actions will be totally dependent on your situation, but I suggest a few areas that I find to be crucial for any modernisation. If you already have them under control, that’s great news and a promising prediction for a smooth process! If you don’t have them under control, the work ahead is probably much harder. But you will also have a very good effect in these areas for the rest of the life of the system. The work will also mature your teams and that investment typically has an even longer lifespan. Doing the work is not throwing money into the sea, it’s rather an investment in your future.
The following are crucial and generic areas you need to have in place. It’s good to have them in place before you make changes. This will make all other changes easier and more risk-free. Without them… Well, then there is quite a lot of work to be done, but just get started!
- Full test automation With this in place, you will run automatic tests to be confident that the system is not broken. This greatly decreases the risk of changes and totally affects how you can work with your software.
- Full observability Even if you have very good test automation in place, there is still the possibility for problems to make their way through the safety net. Therefore you need to have great visibility into what is going on in the software when it runs in production. That’s extremely important so that when there is a problem, it’s quickly sorted out and solved.
- Fully automatic deploy You will probably make quite a lot of changes to your software during the transformation initiative. You want those changes to be deployed as quickly after they were made as possible, and you want each deployment to be done in the same way. For reasons of frequency and repetitiveness, automation is crucial. This deploy pipeline you’ve had or will add probably has a somewhat tight coupling to your environment. That’s also something that has to change, but having that automation is still very positive. It just has to be changed for the new target environment at a certain point.
Action plan
Now it’s time for action! At the time of writing, the risk has skyrocketed, but we are not in panic. We can and should take action in a way that is the best for us. Thereby slowly but surely take steps towards the vision. I suggest that you take the necessary steps in your current development environment and push the changes to your current runtime environment. By doing that, you will be able to continue business as usual with as little friction as possible. You will also start benefiting from the work immediately since you will run the changed and improved code now rather than at some point in the future when you do the move.
This is very much in line with the ideas of “deliver early and often” and “taking baby steps”. The opposite is the “big bang”, and we are not after creating a new universe, only moving our system in a responsible and reliable way. If you decide to do the changes in the new environment, you might find yourself doing a lot of work before the system runs again. It’s like working in the dark, with less-than-wanted feedback.
Another advantage of making the changes in your current environment rather than in the target environment is that you buy yourself time to get used to the target environment, by trying it out. And that is without the pressure of making sure the system runs well in that new and “unknown” environment. You will get to that pressure point, but it’s better to grow into it rather than making a single jump there.
Note
Super weird things are happening every day right now so I might be wrong about the possibility of moving slowly. I actually lean toward panic mode rather, but it’s for you to decide. If so, change the steps accordingly
Still the same feeling of being trapped?
Hopefully you are starting to get a feeling of how you could get out of the trap. That said, a friend said that law wins (I couldn’t make myself write “trumps”) over tech and politics wins over law… That might be the case, but with good tech, you will be flexible and can manoeuver. That’s of course a good thing even if you later decide that you don’t need to move your system.
Trust me, there is hope! After all, the system is just software. “Soft” means that it’s changeable. I think it was Grady Booch who said, 1s and 0s is the best building material in the world.
@Community, please chime in and help out! What did you think about as helpful to tell non-developers at all organisations that would like to move their crucial core systems?
This article was originally published on LinkedIn. Join the discussion and read more here.
The second part in this mini series is found here: https://www.linkedin.com/pulse/feeling-trapped-us-cloud-common-modernisation-pitfalls-jimmy-nilsson-9w4vf/