Docker / Kubernetes

[MirageF1]

Hi folks

thought I'd start a thread around the increasing demand and need for Enterprise containerisation and orchestration skills.
Would welcome any insight and personal experience from those of you already involved or working in this area.

I come from a 3 tier architecture dev environment, but have recently been thinking of seriously entering this area.
Have some basic understanding around concepts and managed to install Docker under WIndows WSL2 ( with Hyper-V) as well as on Ubuntu 20.04 LTS Virtualbox machine (Under non Hyper-V Windows host) and run a few simple examples of single/multi containers/persistent disks/networks, etc

My hardware is limited to a dual core i5 laptop, 12GB RAM, so realise the limitations in what I can realistically achieve, but hoping to get some of the basics under my belt at this stage.

The official docker site documentation and examples, Google, tech forums have been my starting point and have started reading the first few chapters of KUBERNETES_AND_DOCKER_AN_ENTERPRISE_GUIDE which very quickly starts to become very in depth and technical...

Notwithstanding all the complexities of this environment, I really enjoy challenge and broad mix of technologies and skills that are needed for this role (some of which I have from my nearly 30 years as software engineer/dev/admin ) and as this is still relatively early days with skills in fairly short supply am hoping to get a foot through the front door in the foreseeable future landing a role in this ever growing area.

 

[MirageF1]

Didn't expect loads of replies, but not 1?!

Either not many forumites involved in this area or more likely shows the current state of practical involvement locally.

Great emerging area of opportunity, less competition too.

 

[Johnatan56]

Not sure exactly what feedback you want?
We use it for any .Net Core app, most dependencies like Rabbit are running in linux containers. It's better to use than setting directly up on machine as environment can be made to have same dependencies etc. as production, so if works on local/staging it will work on production (in terms of package dependencies).

Again, there isn't really much else to say, it's a tool that works well once set up and then you can kind of forget about it. Have never heard of a place they haven't experimented with it post ~2017 if it's a dev-centric company (2015 for web dev. if web dev and not using it, you should run away from that company).

 

[InvisibleJim]

I did Brett Fisher's Udemy course on Docker and Kubernetes over Christmas - quite comprehensive on the Docker side, Kubernetes was a bit more introductory. I've been reasonably au fait with Docker for a while but still scratching the surface of Kubernetes and it was a worthwhile course for me.

Your hardware is perfectly adequate for experimenting with Docker. Check out Multipass from Canonical - I used it to speed up provisioning a 3 node Ubuntu Swarm in Hyper V on my Desktop pc which is a similar spec to yours.

 

[Benedict A55h0le not]

I avoid Docker and Kubernetes, I prefer automation over conguration, I prefer serverless. I would 1st look at reasons why not to use Azure Functions instead.

Regarding the local dev environment, yes its a problem. I am developing apps that are supposed to harness the limitless power of the cloud on my laptop. Needless to say local dev stress testing is pointless. Overall the dev experience with functions is awesome, the Azure local emulator is great for dev.

 

[PsyWulf]

Didn't expect loads of replies, but not 1?!

Either not many forumites involved in this area or more likely shows the current state of practical involvement locally.

Great emerging area of opportunity, less competition too.

What were you fishing for? Sounded like a blog post or facebook post

Want a like?

 

[Benedict A55h0le not]

Didn't expect loads of replies, but not 1?!

Either not many forumites involved in this area or more likely shows the current state of practical involvement locally.

Great emerging area of opportunity, less competition too.

I see great confusion amongst devs on how to view new technology like micro-services and other cloudy things. Most are like "you don`t need it" or "it only fits some niche need". Many devs that are used to the monolithic architecture just will not accept that their skills are outdated, they still have their legacy apps to cling to to claim as the platform of the future. So yes there is great oppertunity if you can adopt new technology with an open mind.

I think this is the underlying issue, the cloud has brought forth the end of the monolithic architecture as we know it. All that monolithic architecture knowledge is just not worth much anymore.

 

[Chiller89]

Having worked exclusively in Docker based environments for a few years, as well as primarily cloud based environments including Azure and AWS. And having worked with Kubernetes in production for a while, I think the first important step is to realise that Kubernetes and Docker are not alternatives of each other. They compliment and leverage each other, yes, however they are not competing technologies. In fact for some time you had to leverage Docker under the hood in Kubernetes clusters. (Note I am talking about Docker specifically here, not Docker Swarm).

I also believe there is a large space for more understanding and information around why this is so useful. I tend to see a lot of people mis-interpreting the usage and advantage of leveraging Docker based environments, especially in the cloud world where automation, scalability and resiliency are of high importance.

A particular area of interest and understanding which I see as quite relevant is understanding the ephemeral nature of Docker containers and their volumes, and having proper consideration for where, when and how exactly you need to persist data. At the core, containers are designed to be ephemeral, however the usage and understanding (or lack thereof) of them and why this is the case, often results in the effectiveness of using this design being somewhat lost in the long run.

I would say it is vastly more important to understand containerisation, what a container is and how it operates, rather than understanding Docker specifically. There are multiple containerisation technologies (Docker being one of them), as well as orchestration technologies (Kubernetes being one of them) but they all attempt to achieve a fundamental concept, rather than an extremely specific use case.

One great page that I've reverted back to quite a few times, is this: https://www.docker.com/resources/what-container however I usually try to take the information in a generic way, and exclude the fact that you are reading a Docker branded page at all in the beginning if you can.

Obviously some personal opinion here from working with these for a while, so take it with a pinch of salt

 

[Johnatan56]

I avoid Docker and Kubernetes, I prefer automation over conguration, I prefer serverless. I would 1st look at reasons why not to use Azure Functions instead.

Regarding the local dev environment, yes its a problem. I am developing apps that are supposed to harness the limitless power of the cloud on my laptop. Needless to say local dev stress testing is pointless. Overall the dev experience with functions is awesome, the Azure local emulator is great for dev.

There are plenty of good cases for monolithic design, completely depends on what you're working on, saying it's good or bad doesn't work, it's always use-case dependent.

EDIT:
Meant to quote:

I see great confusion amongst devs on how to view new technology like micro-services and other cloudy things. Most are like "you don`t need it" or "it only fits some niche need". Many devs that are used to the monolithic architecture just will not accept that their skills are outdated, they still have their legacy apps to cling to to claim as the platform of the future. So yes there is great oppertunity if you can adopt new technology with an open mind.

I think this is the underlying issue, the cloud has brought forth the end of the monolithic architecture as we know it. All that monolithic architecture knowledge is just not worth much anymore.

 

[Benedict A55h0le not]

There are plenty of good cases for monolithic design, completely depends on what you're working on, saying it's good or bad doesn't work, it's always use-case dependent.

Devs always say this, and then in reply I always ask to give examples. Can you give examples? Saying it is use-case dependent is not a good view to have. Rather look for reasons not to use a new tech in what you want to do. From what I have seen in business, its never a "use-case" thing, its always the same - the system should be rebuilt but the code base is so huge that its impossible. I am yet to find that use-case that necesitates monolithic design. Obviously a client app UI part can still be monolithic, but the overall architecture can be SOA.

 

[Hamster]

Either not many forumites involved in this area or more likely shows the current state of practical involvement locally

We're all busy with emigration.

 

[s0lar]

Devs always say this, and then in reply I always ask to give examples. Can you give examples? Saying it is use-case dependent is not a good view to have. Rather look for reasons not to use a new tech in what you want to do. From what I have seen in business, its never a "use-case" thing, its always the same - the system should be rebuilt but the code base is so huge that its impossible. I am yet to find that use-case that necesitates monolithic design. Obviously a client app UI part can still be monolithic, but the overall architecture can be SOA.

Not sure what hole guys are stuck in, or just fear of change.

K8s is pretty old hat right now. Totally not the new and shiny, been running multiple clusters in production for just shy of 4 years now.

Biggest caveat is guys who don't really want to transition end up with a distributed monolith. Which is worse than a monolith, 9 time out of 10 this is the reason guys say a monolith is better suited to our "stack". Seen it with my own eyes, in production

 

[SHL]

Didn't expect loads of replies, but not 1?!

Either not many forumites involved in this area or more likely shows the current state of practical involvement locally.

Great emerging area of opportunity, less competition too.

Done

 

[Hamster]

distributed monolith. Which is worse than a monolith

Depends on the monolith. I worked on an environment for three years and we could go from commit to prod in about 5-10 minutes. Best environment I ever worked on

But then I've seen the more common ****ed up other side of it where a prod release takes weeks (banks man).

Anyway, a badly done or too granular micro service environment is just as big a nightmare.

 

[Benedict A55h0le not]

Not sure what hole guys are stuck in, or just fear of change.

K8s is pretty old hat right now. Totally not the new and shiny, been running multiple clusters in production for just shy of 4 years now.

Biggest caveat is guys who don't really want to transition end up with a distributed monolith. Which is worse than a monolith, 9 time out of 10 this is the reason guys say a monolith is better suited to our "stack". Seen it with my own eyes, in production

Old school architects who are in bed with other non-IT directors. They kinda promised to deliver a future ready platform with a monolith. Then deved a huge monolith for 10 years, ignores the new cloud tech, and now a huge system needs to be re-developed in an archtecture unknown to the architect. This is it means being stuck. Too much code and the architect says all is good but in reality the competitor already has better technology and have completed their digital transformation and is ready for the future.

I have seen an old school architect build microservices on his own with exe processes, only thing was that these microservices all needed to be on the same VM so when they scaled out because the server was under load, it just added more load to the server. Then in came plan B which was to deploy the entire monolith server to serarate VMs as a "micro-service" setup.

 

[Johnatan56]

Devs always say this, and then in reply I always ask to give examples. Can you give examples? Saying it is use-case dependent is not a good view to have. Rather look for reasons not to use a new tech in what you want to do. From what I have seen in business, its never a "use-case" thing, its always the same - the system should be rebuilt but the code base is so huge that its impossible. I am yet to find that use-case that necesitates monolithic design. Obviously a client app UI part can still be monolithic, but the overall architecture can be SOA.

Tbh, I don't think there are many cases anymore, but you can do so with something like a Blazor system, or if you have a node.js server with servcer side rendering, etc.

I would really only consider it for applications where you don't have enough services that it makes sense to build it into a microservice and you need to be able to easily maintain it in terms of everything uses the same dependencies (e.g. full .Net FW stack or PHP dashboard, etc.).

So yes, would say edge case, most modern systems where you want to have e.g. web app and a rest service, then you should definitely not do so.

So it's for those smaller cases that are just not big enough to justify having to set up and manage multiple projects, especially for 1/2 man projects. Anything bigger than that probably needs to scale and has multiple people working on it, so better to split it out into proper services. Proper service design is another great topic to delve into, I prefer service > dll pretty much any day, but know others who will always go dll route as "performance".

 

[Sinbad]

My biggest gripe with Kubernetes is that it feels like it's incomplete. Sure, there's a lot of functionality there, but to actually make use of it in a manageable and sustainable way you need something like rancher or openshift.
The whole paradigm of "create a yaml file then run kubectl -f" is absolutely barbaric.
If nothing else, they could at least templatise the damn thing... Run a command to create a PVC, for example, and it pops up a text editor with the yaml prepopulated in it so you can just fill in the values....

 

[Johnatan56]

My biggest gripe with Kubernetes is that it feels like it's incomplete. Sure, there's a lot of functionality there, but to actually make use of it in a manageable and sustainable way you need something like rancher or openshift.
The whole paradigm of "create a yaml file then run kubectl -f" is absolutely barbaric.
If nothing else, they could at least templatise the damn thing... Run a command to create a PVC, for example, and it pops up a text editor with the yaml prepopulated in it so you can just fill in the values....

In terms of the main kubernetes or pods? Since pods have templates you can define: This: https://kubernetes.io/docs/concepts/workloads/pods/#pod-templates ?

 

[Sinbad]

In terms of the main kubernetes or pods? Since pods have templates you can define: This: https://kubernetes.io/docs/concepts/workloads/pods/#pod-templates ?

Everything except pods. Basically the cluster admin and maintenance.

 

[B-1]

Didn't expect loads of replies, but not 1?!

Either not many forumites involved in this area or more likely shows the current state of practical involvement locally.

Great emerging area of opportunity, less competition too.

Its not exactly a light hearted discussion that's easy to do in a forum format. I think if you narrow things down a lot you will get much more interaction.