Raj S retweeted
Containers Explained ( Zero To Hero )
What They Are, Why We Use Them, and How Docker Works.
Ever heard developers say:
"It works on my machine!"
You build an application, test it locally, and everything works.
But when you deploy it to another server, it crashes because of different runtime versions, missing dependencies, or environment configurations.
Containers help solve this problem.
Here's everything you need to know about containers, Docker, and sidecar containers.
1/ What Is a Container?
-> A container is a lightweight, portable, and isolated environment used to run an application.
It packages everything an application needs:
- Application code
- Runtime (Node.js, Python, Java)
- Libraries and dependencies
- Required tools
- Configuration
Think of it as a ready-to-run environment for your application.
2/ What Problems Do Containers Solve?
Containers provide a consistent runtime across different environments, isolate dependencies to avoid conflicts, package applications to reduce the โworks on my machineโ problem, simplify deployments through reusable images, use fewer resources than traditional VMs in many workloads, and make applications easier to move between compatible environments.
Containers make applications easier to build, test, deploy, and scale.
3/ How Do Containers Work?
Understand these three terms:
๐ข Dockerfile: A text file containing instructions to build an image.
๐ข Docker Image: A read-only template containing application code, runtime, dependencies, and startup instructions.
๐ข Container: A running instance of an image.
The relationship:
Dockerfile โ Docker Image โ Container
4/ How Do We Create a Container Using Docker?
Let's take a Node.js application.
Create a Dockerfile:
FROM node:alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY . .
EXPOSE 3000
CMD ["node", "server.js"]
Build the image:
~ docker build -t my-app:1.0 .
Run the container:
~ docker run -d \
--name my-container \
-p 3000:3000 my-app:1.0
Your application is now running inside a container!
5/ How Do We Share Container Images?
Imagine you built an image on your laptop. How will another developer or production server access it?
Use a container registry such as:
- Docker Hub
- Amazon ECR
- GitHub Container Registry
Push your image:
~ docker tag my-app:1.0 username/my-app:1.0
docker push username/my-app:1.0
On another machine:
~ docker pull username/my-app:1.0
docker run -d -p 3000:3000 username/my-app:1.0
Build once, share the image, and run it in compatible environments.
6/ Containers vs Virtual Machines
A major difference:
Virtual Machines
- Run a separate guest OS.
- Include their own OS kernel.
- Generally require more resources.
Containers
- Share the host OS kernel.
- Provide process and resource isolation.
- Are generally lightweight and start quickly.
Containers are not mini virtual machines.
Note: Docker Desktop on macOS and Windows uses a managed Linux VM to run Linux containers.
7/ What Is a Sidecar Container?
A sidecar is a supporting container that runs alongside the main application container.
Imagine your Java Spring Boot application needs log collection, monitoring, or proxy functionality.
Instead of putting everything inside the application container, you can run supporting containers alongside it.
Example:
Main Application Container
- Runs Java application.
- Handles API requests.
- Executes business logic.
Sidecar Container
- Collects application logs.
- Forwards logs to a central platform.
- Can provide proxying or monitoring.
In Kubernetes, containers in the same Pod share the Pod's network namespace and can share storage through mounted volumes.
They can communicate using localhost.
8/ Sidecar vs Init Container
Don't confuse these two:
Init Container: Runs a setup task and completes before the application container starts.
Sidecar Container: Runs alongside the application and supports it during its lifecycle.
Sidecars are commonly useful for logging, monitoring, networking, and security workloads in Kubernetes.
9/ Do Containers Replace Virtual Machines?
No. Both have their own use cases and are often used together.
For example, on AWS:
Launch an EC2 instance.
Run a container runtime.
Pull your application image from Amazon ECR.
Start your application container.
You can also use managed platforms such as Amazon ECS and Amazon EKS.
A typical deployment workflow:
Developer โ Dockerfile โ Docker Image โ ECR โ ECS/EKS โ Running Application
10/ Important Docker Commands
docker ps # Running containers
docker ps -a # All containers
docker images # List images
docker pull nginx # Pull an image
docker logs <name> # View logs
docker exec -it <name> sh
docker stop <name> # Stop container
docker rm <name> # Remove container
docker rmi <image> # Remove image
NK Thoughts:
Containers changed how we build, package, deploy, and scale applications.
Remember:
- Docker helps build and run containers.
- Container registries store and distribute images.
- Kubernetes orchestrates containers at scale.
- Terraform provisions infrastructure.
- CI/CD automates application delivery.
Understanding containers is one of the most important foundations for anyone learning DevOps, Docker, Kubernetes, and cloud computing.
Save this post for your next DevOps learning session and Repost to share with others.
Raj S retweeted
How GitHub Stores Git Objects at Massive Scale
โ GitHub manages millions of repositories containing billions of Git objects
โ Every commit, file, directory, and tag must be stored, retrieved, and managed efficiently
โ To achieve this, GitHub combines Git's object model with optimized storage infrastructure
โ 1. Git Object Model
โ Git stores repository data as objects
โ The four main Git object types are:
Blob โ Stores file contents
Tree โ Represents directories and file structures
Commit โ Records snapshots and repository history
Tag โ References specific commits or objects
โ Every Git object is identified using a cryptographic hash
โ 2. Content-Addressable Storage
โ Git identifies objects using their content hashes
โ Identical content can share the same object
โ Objects can be retrieved using their unique identifiers
โ This makes object storage predictable and efficient
โ 3. Object Compression
โ Git compresses objects to reduce storage consumption
โ Packfiles combine multiple objects into a single file
โ Delta compression stores differences between similar objects
โ This reduces duplicated data and storage requirements
โ 4. Packfile Generation
โ Loose objects can accumulate as developers commit changes
โ Git periodically groups objects into packfiles
โ Packfiles improve storage efficiency
โ Index files help Git locate individual objects quickly
โ 5. Repository Storage Infrastructure
โ Millions of repositories require scalable storage infrastructure
โ Repository data is organized across storage systems
โ Storage operations are optimized for high-volume access
โ Infrastructure must support cloning, fetching, and pushing
โ 6. Object Retrieval
โ A developer requests a repository or specific Git objects
โ Git identifies the required objects
โ Packfiles and indexes help locate the requested data
โ The server sends the required objects to the client
โ Git reconstructs the repository data locally
โ 7. Caching and Performance
โ Frequently accessed repository data can be cached
โ Caching reduces repeated storage operations
โ Popular repositories benefit from faster access
โ Optimized retrieval reduces infrastructure workload
โ 8. Data Integrity and Reliability
โ Git object hashes help verify content integrity
โ Storage systems require monitoring and failure recovery
โ Replication and backups help protect important repository data
โ Reliability mechanisms reduce the impact of infrastructure failures
โ 9. Continuous Storage Optimization
โ Repository sizes grow as developers contribute code
โ Storage systems need continuous optimization
โ Packfile maintenance reduces unnecessary storage overhead
โ Infrastructure evolves to support increasing repository workloads
โ The Core Idea
โ Represent data as Git objects
โ Identify objects using cryptographic hashes
โ Compress objects efficiently
โ Combine objects into packfiles
โ Index objects for faster retrieval
โ Distribute storage infrastructure
โ Cache frequently accessed data
โ Protect data integrity
โ Continuously optimize storage
โ GitHub scales Git storage by combining content-addressable objects, compression, packfiles, indexing, caching, and distributed infrastructure.
โ Git & GitHub Ebook
โ Learn Git and GitHub through practical workflows, version control, branching, collaboration, and real-world developer workflows.
โ codewithdhanian.gumroad.com/โฆ
Raj S retweeted
A beginner's guide to Spring Boot performance tuning
vladmihalcea.com/spring-bootโฆ
Raj S retweeted
Interviewed a Senior Backend Engineer today.
Knew Redis, SQL, indexing, all of it.
Then I asked:
You have 1 billion registered usernames.
A user types a new username on the signup form.
You must say "taken" or "available"
within 50ms, on every keystroke.
The engineer said: query the database.
1 billion rows.
Every keystroke.
From millions of users.
Next idea: cache every username in Redis.
1 billion strings is tens of gigabytes
before overhead.
Silence.
How would you design it? ๐
Raj S retweeted
How to spot perfromance problems with the Hibernate slow query log
vladmihalcea.com/hibernate-sโฆ
Raj S retweeted
A Senior Platform Engineer candidate had 8 years of experience.
His resume mentioned Kubernetes, EKS, Terraform, Helm, Karpenter, and platform automation.
He told me he had managed Kubernetes clusters running hundreds of microservices.
So I gave him a scenario.
Problem:
Your Kubernetes cluster has 20 worker nodes.
A sudden traffic spike causes the Horizontal Pod Autoscaler to increase replicas from 10 to 50.
But 30 new Pods remain Pending.
The monitoring dashboard shows plenty of unused CPU and memory across the cluster.
Candidate's answer:
I'll increase the node count and configure Cluster Autoscaler.
Sounds reasonable.
But then I asked:
The cluster has enough aggregate CPU and memory to run the additional Pods.
Still, Kubernetes refuses to schedule them.
Why?
He replied:
"Maybe the scheduler is overloaded. I'll restart it."
So I gave him more information.
The Pending Pods request 4 CPU cores each.
Most nodes have only 2 CPU cores available individually.
What happens now?
He paused.
Then suggested increasing the HPA maximum replicas.
So I asked:
Does Kubernetes schedule a Pod based on aggregate cluster resources?
How do resource requests influence scheduling?
What happens when topology spread constraints cannot be satisfied?
How do taints, tolerations, and node affinity affect placement?
How would you diagnose a Pending Pod using scheduler events?
He knew how to scale deployments.
But he struggled to explain how Kubernetes decides where a Pod can run.
Having 100 free CPU cores across a cluster doesn't mean Kubernetes can schedule a Pod requesting 4 cores.
A Pod must fit on an eligible node.
Kubernetes scheduling is about placement constraints, not just total resources.
Would you trust this engineer to design autoscaling for a production platform?
I asked a Senior AWS Architect with 11 years of experience to design a disaster recovery solution.
His resume included AWS Solutions Architect Professional, Terraform, multi-region deployments, and enterprise cloud migrations.
The requirement was simple:
An enterprise application must recover within 15 minutes of a regional outage.
Maximum acceptable data loss: 5 minutes.
He proposed:
Primary infrastructure in us-east-1.
Standby infrastructure in us-west-2.
Cross-region database replication.
Route 53 failover routing.
Infrastructure managed through Terraform.
A good starting point.
Then I asked one question.
"Your primary AWS region goes down at 2:00 AM.
At 2:03 AM, the standby application is running.
At 2:05 AM, DNS has switched.
But customers still cannot log in."
What's your next step?
He said:
"I'll check Route 53. Maybe DNS hasn't propagated."
I told him DNS was working.
The application was reachable.
The database was healthy.
But authentication was failing.
He suggested restarting the authentication service.
Then I revealed the issue.
The application depended on an identity service hosted only in the failed region.
His disaster recovery architecture replicated the infrastructure.
But not every dependency required to operate the application.
So I asked:
How do you identify cross-region dependencies?
Where are secrets and encryption keys stored?
What happens to sessions during regional failover?
How would you validate RPO and RTO?
When was your last actual DR drill?
He could explain replication.
But he couldn't explain how to prove the application would recover.
A DR architecture diagram is not a disaster recovery strategy.
Your RTO and RPO are only targets until you've tested them under realistic failure conditions.
Would you sign off on a DR solution that has never survived a full failover test?
Raj S retweeted
Interviewed a Senior Backend Engineer today.
Knew SQL, transactions, locking, all of it.
Then I asked:
Two admins open the same customer record.
Admin A changes the phone number.
Admin B changes the address.
Both click Save, 5 seconds apart.
Admin B's save sends the whole record,
including the OLD phone number.
Admin A's change is silently gone.
No error. No warning. Nobody notices.
The engineer said: use a transaction.
Both transactions commit fine.
The overwrite still happens.
Silence.
How do you stop the lost update? ๐
Raj S retweeted
๐ฆ๐๐ฎ๐๐ฒ๐ณ๐๐น ๐๐ ๐ฆ๐๐ฎ๐๐ฒ๐น๐ฒ๐๐ ๐๐ฒ๐๐ถ๐ด๐ป.
Do you know the differences?
A ๐๐๐ฎ๐๐ฒ๐น๐ฒ๐๐ service keeps nothing between requests. Each request carries what the server needs, like a token or an ID, so any instance can handle it. Scaling is simple: add instances behind a load balancer, and losing one loses nothing.
A ๐๐๐ฎ๐๐ฒ๐ณ๐๐น service remembers. A session, an open connection or a shopping cart lives on the server between requests. Interactions get richer, but requests now have to reach the right instance, and losing it means losing that state.
If you're building AI agents that need to remember, here's a great breakdown โ lucode.co/ai-agent-memory-orโฆ
Most systems get both by keeping services stateless and moving state into a shared store, a database or cache every instance can reach.
Stateless = easy to scale. Stateful = continuity. Externalised state = both.
LLMs are stateless too. Every call starts fresh, and the model only knows what's in the prompt. An agent that "remembers" is doing the same trick: keeping its state outside the model and loading it back in.
But an agent's state is harder than a shopping cart.
The current conversation has to stay small enough to fit.
What a user said last month has to be found by meaning, not by key.
And one user's memories can never surface for another.
That changes how agent memory should be designed.
What else would you add?
โโ
๐พ Save it for later.
๐ Thanks to @OracleDevs for sponsoring this post.
โ Follow Nikki Siapno to improve at system design and AI engineering.
Raj S retweeted
YouTube video Platform Architecture
A Product Manager, a dev, a code reviewer and a bug fixer walk into a chat.
Switch Buddy sets up all four AI agents in one conversation. Then each one introduces itself.
Connect GitHub and they act as you. Iโd give the reviewer read access and the bug fixer write access.
Install the Switch Console.
How to Configure LAyer 3 Switch SVI on Cisco? | Example Configuration | IPCisco.com
.
Get Packet Tracer Labs: ipcisco.com/cisco-packet-traโฆ
#network #cisco #ccna #cisconetworking #ciscocertification
Raj S retweeted
Goldmine for Software Engineers! ๐
If you want to learn System Design and/or AI Engineering, then bookmark this repo right now.
github.com/systemdesign42/syโฆ
Raj S retweeted
Nobody uses git grep in real projects.

ALT Git Commands Cheat Sheet shared by @PythonDvz organized into 6 sections: Setup with git init and git clone, Daily Workflow with git status, git add, git diff --staged and git commit, Branches with git switch -c and git merge, Remote with git fetch, git pull and git push -u origin, Undo Recover with git restore --staged, git revert, git reset --soft and git stash pop, plus Visualize Search with git log --oneline --graph, git blame and git grep.
Raj S retweeted
Latency Numbers Every Developer Should Know
CPU & Memory:
- Register access: ~0.3 ns
- L1 cache: ~1 ns
- L2 cache: ~4 ns
- L3 cache: ~20 ns
- RAM access: ~100 ns
Storage:
- SSD read: ~100 ฮผs
- Hard disk seek: ~10 ms
Network:
- Same datacenter round trip: ~500 ฮผs
- California to Netherlands round trip: ~150 ms
Note: These are illustrative latency estimates. Actual numbers vary based on hardware, workload, access patterns, and network conditions.
Top 5 Ways to Reduce Latency:
1. Caching: Keep frequently accessed data closer to the CPU or application.
2. Database optimization: Use proper indexing and avoid unnecessary disk I/O.
3. Batching: Combine multiple operations to reduce network round trips.
4. Asynchronous processing: Avoid blocking threads while waiting for slow I/O operations.
5. Data locality: Keep related data and services close to minimize network latency.
โป๏ธ Repost to help others learn this.
Raj S retweeted
A Senior DevOps Engineer proudly told me his CI/CD pipelines had a 100% success rate.
His resume listed GitHub Actions, Jenkins, ArgoCD, Helm, Terraform, and Kubernetes.
So I asked:
"What happens when your pipeline succeeds but the deployment fails?"
He looked confused.
"If the pipeline passes, the deployment should be successful."
I gave him a scenario.
Your team deploys a new version of a payment service.
โ
Unit tests pass.
โ
Docker image builds successfully.
โ
Security scans pass.
โ
Helm deployment completes.
โ
Kubernetes Pods report Ready.
Five minutes later, payment failures increase by 40%.
Your pipeline is green. Production is broken.
What do you do?
He answered:
"I'll configure automatic rollback if the deployment fails."
But the deployment didn't fail.
The application was running.
The Pods were healthy.
The problem was in the business logic.
So I asked:
How do you detect failures that readiness probes miss?
What metrics should control a canary rollout?
How do you compare a new version against the stable version?
What happens if your database migration isn't backward compatible?
How do you prevent bad releases from reaching 100% of users?
He knew how to automate deployments.
But he struggled to explain how to validate their real-world impact.
A successful CI/CD pipeline doesn't mean a successful production release.
Senior DevOps Engineers don't just automate deployments.
They engineer safe ways to discover that a deployment is wrong.
Would you approve a production pipeline that only checks whether Kubernetes reports healthy Pods?
Raj S retweeted
A beginner's guide to Hibernate ON CONFLICT DO clause
vladmihalcea.com/hibernate-oโฆ
Raj S retweeted
Docker runs containers; Kubernetes orchestrates them at scale.
โ Docker โ Containerization
โก Kubernetes โ Orchestration
โข Docker โ Runs containers
โฃ Kubernetes โ Deploys and scales Pods
โค Docker โ Container management
Both are core to modern DevOps.
#Docker #Kubernetes #AI
Raj S retweeted
๐ข๐๐๐๐ต ๐๐ ๐๐ฃ๐ ๐๐ฒ๐ ๐๐ ๐๐ฎ๐๐ถ๐ฐ ๐๐๐๐ต ๐๐ ๐บ๐ง๐๐ฆ.
Do you know the differences?
They all help secure access, but what they actually verify, and how they do it, is very different.
๐ข๐๐๐๐ต ๐ฎ.๐ฌ uses scoped access tokens to let applications access resources without sharing user passwords. Tokens can expire and access can be revoked.
๐๐ฃ๐ ๐๐ฒ๐๐ use a shared secret to identify and authenticate the calling application. They're simple to implement, but the key itself doesn't prove who holds it.
๐๐ฎ๐๐ถ๐ฐ ๐๐๐๐ต sends a username and password with every request. It's straightforward, but credentials must be protected with HTTPS.
๐บ๐ง๐๐ฆ uses certificates to authenticate both the client and server during the TLS handshake. It's useful for service-to-service communication where both sides need to verify identity.
But choosing an authentication method is only part of the problem. You also need to manage credentials throughout their lifecycle.
Take CI/CD pipelines. A deployment might depend on a long-lived API key or deploy key stored in its secrets.
Even if it's stored securely, that credential can still be leaked, copied, or forgotten. And rotating it can break deployments that depend on it.
One approach is to replace static credentials with ๐๐ต๐ผ๐ฟ๐-๐น๐ถ๐๐ฒ๐ฑ, ๐ถ๐ฑ๐ฒ๐ป๐๐ถ๐๐-๐ฏ๐ฎ๐๐ฒ๐ฑ ๐ฎ๐ฐ๐ฐ๐ฒ๐๐.
Instead of retrieving a stored secret, the pipeline proves its identity and receives a credential scoped to what it needs. That credential expires automatically.
Teleport Machine ID implements this approach by letting CI/CD runners use their platform identity to obtain short-lived certificates instead of storing deploy keys.
This reduces the need to manage long-lived credentials while keeping access tied to specific machine identities.
๐๐ฒ๐ฎ๐ฟ๐ป ๐ต๐ผ๐ ๐๐ผ ๐ฒ๐น๐ถ๐บ๐ถ๐ป๐ฎ๐๐ฒ ๐๐๐ฎ๐๐ถ๐ฐ ๐ฐ๐ฟ๐ฒ๐ฑ๐ฒ๐ป๐๐ถ๐ฎ๐น๐ โ lucode.co/teleport-machine-wโฆ
What else would you add?
โโ
๐พ Save it for later.
๐ Thanks to @goteleport for sponsoring this post.
โ Follow Nikki Siapno to improve at system design and AI engineering.
































