How much experience do you have with Docker? I am not being snarky -- all the things you say sounds like something I would have said until I pulled myself together and learned how to use it properly. And granted, that did take a bit of effort but now my perspective is that Docker is extraordinarily easy to work with.
A Dockerfile is almost exactly like your requirements.txt file, only it works for everything. Need Imagemagick installed on your server? Just add it to the Dockerfile. Need to run a pre-start tool that was written in Ruby? A line or two in the Dockerfile can add that. And if it turns out that you don't want it after all, you just remove the lines and feel confident that no trace is left behind.
I don't have a lot of experience with Docker, but it's not what's holding me back. Here are some of my pain points:
1) Having to "docker exec" to get a shell in a running container vs just running the command (in dev) or sshing into the server (in prod).
2) Tests taking 3:30 min to run with docker vs 30 secs without. And then of course, you don't just run tests once but many times, leading to tens of minutes on a given day where I'm just twisting my thumbs.
3) Having to even spend time learning how to debug Python code inside a locally running docker container.
4) Having to deal with disk space issues caused by old docker images on my 500gb hdd.
All while not deriving any value from it in my projects.
> 2) Tests taking 3:30 min to run with docker vs 30 secs without.
This is not a problem with docker, but your environment. Tests should run at the same speed and initial time with or without docker (with an exception for changes that update dependency list - that will take the time for the initial build).
Things to check: Are you installing dependencies before adding the app? For development are you mounting the app instead of building a new image each time?
Give Docker a go on a project and you'll see its utility:
1. If you're doing this, then you're likely using docker incorrectly. Your container should be run from images that are automatically deployed. If there's an issue, fix local and deploy image.
2. I agree, that sounds dreadful. We use pyenv and poetry for local development and docker for deployments. That would address your issue. We of course do not use pyenv or poetry inside the docker image. Hopefully that helps clarify potential real world use.
3. Your should not be debugging a live deployment. Debugging local container is straight forward in VSCode.
4. This shouldn't be an issue. Prune weekly, and aim for smallest images where possible, e.g., 60MB for python microservice.
The value is in the simplicity of deployment. Need to host a well-known software? Can be up or down no time.
As I wrote at the very top of this thread, i have used Docker in a project that was forced on me, and I still use it in other client projects where I don't have a choice. I don't need to "give it a go" to learn what it's like.
1. First, the infra to automatically deploy again introduces complexity. And waiting for the Docker push to complete and then until the new container is started takes away from my time.
2. Yes, it is a pain.
3. I do what's necessary to fix problems. Sometimes that means looking at production. I use PyCharm not vs code and because I don't find value in Docker for my projects have no incentive to look into how to set up local debugging.
4. "Prune weekly" you say. But it's just another complication I have to deal with when using Docker. What for?
> The value is the simplicity of deployment
I would argue my deployments are simpler than yours. Give me git and ssh and I'm good to go. No need for Docker, pushing to some registry, looking at a dashboard or waiting for the image to be deployed. And my setup is much easier to debug.
But it sounds to me like you are coming from a more enterprise-y environment. There, the things you say probably make sense. In my case, I'm a sole developer. Any unnecessary process or tool slows me down and incurs a risk of bugs due to added complexity.
> I would argue my deployments are simpler than yours. Give me git and ssh and I'm good to go.
FYI: the wait time is marginal at best to deploy images and containers. My workflow is the same as yours most of the time for solo projects: SSH into server, git pull (from a "stack" branch that has docker-compose files containing DB and microservices), docker-compose up -d, and because of cached images it takes minimal time to deploy.
--
I agree that as a sole developer it can add initial complexity. As a sole developer myself on prior projects and now on personal projects, I use Docker primarily to streamline my deployment practices.
Another excellent use case of Docker not mentioned elsewhere in this thread is the simplicity of running databases locally, e.g., mongodb, postgres, etc.
Tests running with this difference implies a misconfiguration or a whole lot of difference on setting up the test pipeline. You shouldnt blame it on docker before investigating the issue.
> Having to "docker exec" to get a shell in a running container vs just running the command (in dev) or sshing into the server (in prod).
If this bother you, take a look at some 3rd party docker management ui such as portainer (vanilla docker) or k9s (kubernetes). These tools will let you navigate and launch shell on your container quickly. Very useful if you have tons of apps running in your node.
> Tests taking 3:30 min to run with docker vs 30 secs without. And then of course, you don't just run tests once but many times, leading to tens of minutes on a given day where I'm just twisting my thumbs.
In my case, I don't use docker in development phase. I test in local environment and only build the images when it's ready for deployment.
> Having to even spend time learning how to debug Python code inside a locally running docker container.
I never had to do this anymore. I just hook sentry or newrelic and they'll log exception stack traces that I can use to figure out the issue without live-debugging the app.
> Having to deal with disk space issues caused by old docker images on my 500gb hdd.
Yes, disk usage is one of the drawback of using docker. It's especially suck pruning images on busy servers with spinning rust. On ssd, pruning is not as slow though.
A Dockerfile is almost exactly like your requirements.txt file, only it works for everything. Need Imagemagick installed on your server? Just add it to the Dockerfile. Need to run a pre-start tool that was written in Ruby? A line or two in the Dockerfile can add that. And if it turns out that you don't want it after all, you just remove the lines and feel confident that no trace is left behind.