Lab 2: Compute & Containers

Is step mein tum SSH ke liye deepdive-key naam ka key pair banaoge, aur deepdive-web-sg naam ka security group banaoge jo SSH sirf tumhare IP se aur HTTP sabki taraf se allow karega.

Key pair mein do chaabiyan hoti hain. Public key AWS instance ke andar rakh deta hai, aur private key (.pem file) sirf tumhare paas rehti hai. AWS tumhari private key ki koi copy nahi rakhta, isliye agar woh kho gayi toh tum us instance mein SSH se kabhi nahi ja paoge. Security group instance ke bahar laga hua ek stateful firewall hai. Isme sirf allow rules likhe ja sakte hain, deny rules nahi. "Stateful" ka matlab hai ki jo request andar aayi, uska jawab apne aap bahar jaane diya jaata hai; uske liye alag rule nahi likhna padta.

1. EC2 console mein Key Pairs → Create key pair kholo. Naam deepdive-key rakho, type RSA aur format .pem chuno, aur Create dabao. Private key ki file tumhare computer pe download ho jaayegi.
2. Terminal mein mv ~/Downloads/deepdive-key.pem ~/.ssh/ && chmod 400 ~/.ssh/deepdive-key.pem chalao. chmod 400 isliye zaroori hai kyunki SSH aisi key ko use karne se mana kar deta hai jise koi bhi padh sakta ho.
3. EC2 console mein Security Groups → Create kholo. Naam deepdive-web-sg rakho aur default VPC chuno. Inbound rules mein ek SSH rule jodo jiska source My IP ho, aur ek HTTP rule jodo jiska source Anywhere-IPv4 ho. Phir Create dabao.

• ls -l ~/.ssh/deepdive-key.pem chalane pe permission -r-------- dikh rahi hai.
• Security group mein port 22 sirf x.x.x.x/32 (yaani tumhare IP) se khula hai, aur port 80 0.0.0.0/0 se.

PL-065 course se — asli console reference

Key pair aur security-group creation ka wahi console.

Is step mein tum ek Amazon Linux 2023 instance launch karoge jo boot hote hi nginx install karega aur ek aisa web page dikhayega jisme uska instance ID aur Availability Zone likha ho.

User-data ek script hai jo instance ke pehle boot pe root user ke roop mein chalti hai; isko cloud-init naam ka tool chalata hai. User-data ki madad se servers "self-configuring" ban jaate hain. Jab Auto Scaling naye instances banata hai, toh har naya instance apne aap ready ho jaata hai aur kisi ko manually SSH karke setup nahi karna padta.

Pehle neeche wali script ko ek file mein save karo. Yahi file Part B aur Part C mein bhi kaam aayegi.

cat > userdata.sh <<'EOF'
#!/bin/bash
dnf install -y nginx
TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 300")
IID=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id)
AZ=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/availability-zone)
cat > /usr/share/nginx/html/index.html <<HTML
<h1>DEEPDIVE web server</h1>
<p>Instance: $IID</p>
<p>AZ: $AZ</p>
HTML
systemctl enable --now nginx
EOF
cat userdata.sh

1. EC2 console mein Launch instances dabao. Naam deepdive-console rakho, Amazon Linux 2023 AMI chuno, free-tier eligible instance type chuno aur key pair mein deepdive-key select karo. Network settings mein default VPC rehne do, public IP enable rakho, aur Select existing security group chun ke deepdive-web-sg select karo.
2. Storage ko default (8 GiB gp3) hi rehne do.
3. Advanced details kholo, User data wale box mein userdata.sh ka poora content paste karo aur Launch dabao.
4. Do minute baad browser mein http://<Public-IPv4> kholo, jahan Public-IPv4 ki jagah instance ka public IP daalo.

• Browser mein "DEEPDIVE web server" ke saath instance ID aur AZ dikh raha hai.

PL-065 course se — asli console reference

User data box — tumhara script nginx install karega, is screenshot mein mkdir/touch hai.

Apne laptop ke terminal se instance mein SSH karo. PUBLIC_IP ki jagah instance ka public IP daalo:

ssh -i ~/.ssh/deepdive-key.pem ec2-user@<PUBLIC_IP>

Instance ke andar pahunch ke yeh commands chalao, aur har command ka output padh ke samjho ki woh kya bata raha hai:

hostname                       # private DNS name: ip-172-31-x-x.ec2.internal
nproc; free -h                 # vCPU aur RAM — instance type ke hisaab se
lsblk; df -h /                 # root EBS volume (nvme0n1) — Lab 3 mein detail
systemctl status nginx --no-pager
sudo tail -20 /var/log/cloud-init-output.log   # user-data ka output yahan milta hai
curl -s localhost
exit

• SSH se login ho gaya.
• cloud-init-output.log mein nginx install hone ka output dikha.
• Tum samjha sakte ho ki user-data dobara reboot karne pe kyun nahi chalta. (Kyunki woh sirf pehle boot pe chalta hai.)

PL-065 course se — asli console reference

SSH client tab aur terminal se connect hone ka wahi tareeka.

Is step mein tum instance ko stop karke dobara start karoge aur dekhoge ki uske IP addresses ke saath kya hota hai.

1. Pehle instance ka Public IP aur Private IP note kar lo. Phir Instance state → Stop dabao, state "Stopped" hone tak ruko, aur phir Start dabao.
2. Naya public IP note karo aur browser mein us naye IP pe page kholo.

Agar tumhe aisa public IP chahiye jo kabhi na badle, toh uske liye Elastic IP hota hai. Lekin dhyan rakho ki 2024 se AWS har public IPv4 address ka lagbhag $0.005 per ghanta charge karta hai (free tier mein 750 ghante muft milte hain). Production mein fixed IP ki jagah aam taur pe Load Balancer aur DNS ka use hota hai, jo tum Lab 4 mein dekhoge.

• Stop aur start ke baad public IP badal gaya, jabki private IP wahi raha.
• Page naye IP pe bhi chal raha hai, kyunki nginx aur index.html disk pe safe the.

PL-065 course se — asli console reference

Terminate / stop dialogs — stop karne pe public IP badalta hai, yahi is step ka point hai.

Is step mein tum console khole bina, sirf CLI se, bilkul wahi configuration wala instance launch karoge.

AMI ID har region mein alag hota hai, aur Amazon Linux ki nayi image lagbhag har mahine aati hai, isliye kisi AMI ID ko script mein hardcode kar dena galat hai. AWS latest Amazon Linux AMI ka ID ek public SSM parameter mein rakhta hai, jahan se tum hamesha sahi ID le sakte ho. Yahi tareeka CloudFormation aur CDK mein bhi use hota hai.

AMI_ID=$(aws ssm get-parameter \
  --name /aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 \
  --query Parameter.Value --output text)
SG_ID=$(aws ec2 describe-security-groups --group-names deepdive-web-sg \
  --query 'SecurityGroups[0].GroupId' --output text)
echo $AMI_ID $SG_ID

IID=$(aws ec2 run-instances \
  --image-id $AMI_ID --instance-type $TYPE \
  --key-name deepdive-key --security-group-ids $SG_ID \
  --user-data file://userdata.sh \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=deepdive-cli},{Key=Project,Value=deepdive}]' \
  --query 'Instances[0].InstanceId' --output text)
echo $IID

aws ec2 wait instance-running --instance-ids $IID
aws ec2 describe-instances --instance-ids $IID \
  --query 'Reservations[0].Instances[0].[State.Name,InstanceType,Placement.AvailabilityZone,PublicIpAddress]' \
  --output table

Kuch cheezein dhyan dene layak hain. Humne --subnet-id nahi diya, isliye AWS ne default VPC ka default subnet use kiya. --query option JMESPath language use karta hai, jisse bade JSON response mein se sirf kaam ki cheezein nikaali jaati hain; yeh har DevOps script mein aata hai. Aur file://userdata.sh dene pe CLI user-data ko khud base64 mein badal deta hai, jaisa EC2 ko chahiye.

• Table mein instance ka state running aur uska public IP dikh raha hai.
• Ek-do minute baad curl http://PUBLIC_IP chalane pe DEEPDIVE page aa raha hai.

Tags ka asli fayda tab dikhta hai jab bahut saare resources hon. Neeche wali command sirf woh running instances dikhati hai jin pe Project=deepdive tag laga hai, aur phir CLI wale instance ko terminate kar deti hai:

# Project=deepdive wale saare running instances
aws ec2 describe-instances \
  --filters Name=tag:Project,Values=deepdive Name=instance-state-name,Values=running \
  --query 'Reservations[].Instances[].[InstanceId,Tags[?Key==`Name`]|[0].Value,PublicIpAddress]' \
  --output table

aws ec2 terminate-instances --instance-ids $IID

• Filter ne sirf tag wale instances dikhaye.
• CLI se banaya gaya instance terminate ho gaya.

PL-065 course se — asli console reference

AWS CLI se tags lagane ka wahi command pattern (yahan create-tags hai).

Is step mein tum ek launch template banaoge, jo ek naye instance ka reusable blueprint hai. Isme AMI, instance type, security group, key pair, user-data aur IAM role, sab ek jagah save rehte hain.

Auto Scaling group ko jab bhi naya instance banana hota hai, toh use launch template se hi pata chalta hai ki instance kaisa banana hai. Launch templates ke versions bhi hote hain. Agar nayi AMI aaye toh tum version 2 bana sakte ho, aur agar kuch gadbad ho toh wapas version 1 pe ja sakte ho.

1. EC2 console mein Launch Templates → Create launch template kholo. Naam deepdive-web-lt rakho aur "Auto Scaling guidance" pe tick lagao.
2. AMI mein Amazon Linux 2023, instance type mein free-tier type, key pair mein deepdive-key aur security group mein deepdive-web-sg chuno. Subnet mat chuno, kyunki woh Auto Scaling group khud chunega.
3. Advanced details mein IAM instance profile deepdive-ec2-role chuno, User data mein userdata.sh ka content paste karo aur Create dabao.

• Launch template ka version 1 ban gaya hai.

Is step mein tum deepdive-web-asg naam ka Auto Scaling group banaoge, jisme kam se kam 1, normally 2 aur zyada se zyada 3 instances honge. Yeh do AZs mein phaile honge aur CPU ke hisaab se scale karenge.

Auto Scaling group teen kaam karta hai. Pehla, self-healing: agar koi instance mar jaaye toh woh naya bana deta hai. Doosra, scaling: load badhne pe instances badhata hai aur load ghatne pe kam karta hai. Teesra, AZ balancing: woh instances ko alag-alag AZs mein baant ke rakhta hai, taaki ek AZ down ho jaaye tab bhi baaki instances chalte rahein.

1. EC2 console mein Auto Scaling Groups → Create kholo. Naam deepdive-web-asg rakho, template deepdive-web-lt (Latest version) chuno aur Next dabao.
2. Default VPC rehne do aur Availability Zones mein us-east-1a aur us-east-1b ke default subnets chuno, phir Next dabao.
3. Load balancing mein No load balancer chuno (load balancer Lab 4 mein jodenge). Health checks ko EC2 rehne do aur Next dabao.
4. Group size mein Desired 2, Min 1 aur Max 3 rakho. Scaling mein Target tracking scaling policy chuno, metric mein Average CPU utilization aur target value 50 daalo. Next dabao, tag Project=deepdive jodo aur Create karo.

• Do naye instances launch hue hain aur woh alag-alag AZs mein hain. (Instances list mein AZ wala column dekho.)
• Dono instances ke public IP pe DEEPDIVE page alag-alag instance ID dikhata hai.

PL-065 course se — asli console reference

Poora Create Auto Scaling group wizard — yeh L11 (27 Mar) ka live demo hai, values tumhari alag hongi.

1. Auto Scaling group ka ek instance select karo aur use Terminate kar do.
2. Auto Scaling group ka Activity tab kholo. Ek-do minute mein pehle "Terminating..." aur phir "Launching a new EC2 instance..." wali entries dikhengi.
3. Instance management tab mein dekho ki phir se do healthy instances chal rahe hain.

Real life mein bhi bilkul yahi hota hai jab kisi server ka hardware fail hota hai ya koi script galti se instance band kar deta hai. Isliye cloud mein servers ko "cattle, not pets" kaha jaata hai: koi ek server special nahi hota, sab ek hi template se bante hain, aur koi gir jaaye toh uski jagah naya aa jaata hai.

• Activity history mein naya replacement instance launch hota dikha.
• Tumhare kuch kiye bina desired count phir se 2 ho gaya.

PL-065 course se — asli console reference

Self-healing: instance terminate karo, ASG turant naya launch karta hai — yahi is step ka core hai.

Is step mein tum Spot instances ki pichhli keemat nikaloge aur use On-Demand keemat se compare karoge.

Spot instances AWS ki khaali padi capacity hoti hain, jo 60–90% discount pe milti hain. Iski shart yeh hai ki AWS jab chahe do minute ke notice pe woh capacity wapas le sakta hai. Isliye Spot batch jobs, CI runners aur aise workers ke liye perfect hai jinka state kahin aur save hota hai, lekin database ya ek akele server wale app ke liye theek nahi hai. Cost kam karne ke aur bhi tareeke hain: Savings Plans ya Reserved Instances (ek se teen saal ke commitment pe lagbhag 30–70% chhoot), Graviton (ARM processor jo sasta padta hai), aur right-sizing (zaroorat se bade instance ko chhota karna, jiska sujhaav Compute Optimizer deta hai).

aws ec2 describe-spot-price-history --instance-types t3.micro m7i.large \
  --product-descriptions "Linux/UNIX" --max-items 6 \
  --query 'SpotPriceHistory[].[InstanceType,AvailabilityZone,SpotPrice,Timestamp]' --output table

On-Demand keemat (us-east-1, Linux) jaanne ke liye console mein EC2 → Instance Types mein m7i.large ka Pricing section dekho, ya EC2 pricing page kholo. Dono keematon se discount ka percentage khud nikaalo.

Agle do steps mein tum ek asli Spot instance chalaoge aur uska interruption simulate karoge. AWS Batch ka hands-on Part I mein hai.

• Tumne apne notes mein m7i.large ke Spot aur On-Demand ke beech ka discount percentage likha hai.
• Tum do aise workloads bata sakte ho jo Spot pe chalne chahiye, aur do aise jo nahi chalne chahiye.

PL-065 course se — asli console reference

Spot requests page aur Reserved Instances cart — pricing model compare karne ka console.

Is step mein tum CLI se ek Spot instance chalaoge aur dekhoge ki woh On-Demand instance se kaise alag dikhta hai.

SPOT_ID=$(aws ec2 run-instances --image-id $AMI_ID --instance-type $TYPE \
  --security-group-ids $SG_ID --iam-instance-profile Name=deepdive-ec2-role \
  --instance-market-options 'MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate}' \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=deepdive-spot},{Key=Project,Value=deepdive}]' \
  --query 'Instances[0].InstanceId' --output text)
aws ec2 wait instance-running --instance-ids $SPOT_ID

aws ec2 describe-instances --instance-ids $SPOT_ID \
  --query 'Reservations[0].Instances[0].[InstanceLifecycle,SpotInstanceRequestId,Placement.AvailabilityZone]' --output table
aws ec2 describe-spot-instance-requests \
  --query 'SpotInstanceRequests[].[SpotInstanceRequestId,State,Status.Code]' --output table

AMI_ID aur SG_ID variables Part B1 se aa rahe hain; agar tumne naya terminal khola hai toh B1 ki pehli do commands dobara chala lo. Spot instances free tier mein nahi aate, lekin t3.micro Spot lagbhag $0.003 per ghanta padta hai.

Ab Session Manager se instance ke andar jao aur interruption-notice wala endpoint check karo. Abhi yeh 404 dega, jiska matlab hai ki koi interruption scheduled nahi hai:

TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 300")
curl -s -o /dev/null -w "%{http_code}\n" -H "X-aws-ec2-metadata-token: $TOKEN" \
  http://169.254.169.254/latest/meta-data/spot/instance-action

• InstanceLifecycle ki value spot hai, aur spot request active ya fulfilled state mein hai.
• instance-action endpoint 404 de raha hai.

PL-065 course se — asli console reference

Spot Fleet request banane ka wahi form.

Spot pe chalne wale app ko do minute ka notice milte hi saleeke se band hona chahiye, jaise apna kaam save (checkpoint) karna aur load balancer se khud ko hata lena. Yeh sab theek se hota hai ya nahi, yeh test karne ke liye AWS ki Fault Injection Service (FIS) instance ko asli interruption signal bhej sakti hai. Jaan-boojh ke failure paida karke system ko test karne ko "chaos engineering" kehte hain. FIS ka kharcha lagbhag $0.10 per action-minute hai, yaani is test ka lagbhag $0.20.

1. AWS FIS console mein Experiment templates → Create experiment template kholo aur description mein deepdive spot interruption likho.
2. Add action dabao, action ka naam interrupt rakho, action type mein aws:ec2:send-spot-instance-interruptions chuno, durationBeforeInterruption ko 2 minutes rakho aur Save karo.
3. Targets section mein apne aap bane target ko edit karo. Resource type aws:ec2:spot-instance chuno, target method mein Resource tags chuno, tag Name = deepdive-spot daalo, selection mode ALL rakho aur Save karo.
4. Service access mein Create a new role for the experiment template chuno, template create karo, aur phir Start experiment dabao.
5. Session Manager ke shell mein neeche wala loop chalao aur dekho ki 404 ka jawab kab JSON mein badalta hai:

while true; do
  TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 300")
  date +%T; curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/spot/instance-action; echo
  sleep 10
done

Kuch hi der mein output mein {"action": "terminate", "time": "..."} aayega, aur do minute baad tumhara session kat jaayega kyunki instance terminate ho jaayega. EC2 console mein instance ka "state reason" padho. Real applications isi endpoint ko baar-baar check karte hain, ya EventBridge ke "EC2 Spot Instance Interruption Warning" event ko sunte hain, aur notice milte hi saleeke se band hote hain. Agar Auto Scaling group mein mixed instances policy ho, toh Spot capacity chhin jaane pe woh doosre instance types ya AZs se nayi capacity le aata hai.

• Metadata endpoint ka jawab 404 se badal ke {"action": "terminate", ...} ho gaya.
• Instance terminate ho gaya aur uske state reason mein Spot interruption likha hai.
• FIS experiment ka state completed hai.

Auto Scaling group ko Edit karo aur Desired ko 0 aur Min ko 0 kar do, phir Update dabao. Jab saare instances terminate ho jaayein, tab group ko Delete karo. Agar tum instances ko seedha terminate karoge, toh Auto Scaling group unki jagah naye bana dega, jaisa tumne C3 mein dekha tha. Launch template ko rakh sakte ho, kyunki uska koi charge nahi lagta.

• Auto Scaling group delete ho gaya hai aur deepdive-web-asg ka koi instance nahi chal raha.

PL-065 course se — asli console reference

Desired/Min ko 0 karke ASG delete karne ka wahi flow.

Pehle ek bucket banao jisme files upload hone pe Lambda function chalega:

export UP_BUCKET=deepdive-uploads-$ACCOUNT_ID
aws s3 mb s3://$UP_BUCKET

Yahi bucket Lab 5 mein EventBridge ke saath bhi use hogi, isliye ise delete mat karna.

• Bucket ban gayi hai.

Is step mein tum deepdive-s3-logger naam ka Python function banaoge, jo har upload hui file ki details JSON format mein log karega.

Lambda mein tum sirf ek handler function likhte ho. Jab koi event aata hai, AWS us function ko ek chhote, managed micro-VM ke andar chalata hai, aur jab function nahi chal raha hota tab tumhe koi paisa nahi dena padta. Har baar function chalne pe use do cheezein milti hain: ek event, jo ek JSON hai aur batata hai ki kya hua, aur ek context, jisme request ID aur bache hue milliseconds jaisi jaankari hoti hai.

1. Lambda console mein Create function dabao aur Author from scratch chuno. Naam deepdive-s3-logger rakho, runtime Python 3.13 aur architecture x86_64 chuno. Permissions mein Create a new role with basic Lambda permissions chuno aur Create dabao.
2. Code tab mein lambda_function.py ka content neeche wale code se badal do aur Deploy dabao:

import json
import urllib.parse


def lambda_handler(event, context):
    records = event.get("Records", [])
    for record in records:
        bucket = record["s3"]["bucket"]["name"]
        # S3 keys URL-encoded aate hain: "my file.txt" -> "my+file.txt"
        key = urllib.parse.unquote_plus(record["s3"]["object"]["key"])
        size = record["s3"]["object"].get("size", 0)
        print(json.dumps({
            "event": record["eventName"],
            "bucket": bucket,
            "key": key,
            "size_bytes": size,
            "request_id": context.aws_request_id,
        }))
    return {"processed": len(records)}

3. Test tab kholo, Create new event chuno, template mein S3 Put select karo, event ka naam s3put rakho, Save karo aur Test dabao. Output mein {"processed": 1} aur logs mein ek JSON line dikhni chahiye.

• Test event successful raha aur "Function Logs" mein JSON line dikhi.

PL-065 course se — asli console reference

Lambda function create karne aur code editor ka wahi console.

1. Function overview mein Add trigger dabao aur S3 chuno. Bucket mein deepdive-uploads-... select karo, event types mein All object create events chuno, prefix mein incoming/ likho, recursive invocation wali warning ko acknowledge karo aur Add dabao.
2. Configuration → Permissions kholo aur neeche Resource-based policy statements dekho. Wahan dikhega ki S3 service ko is function ko chalane ki permission de di gayi hai.

Ab do files upload karo, ek incoming/ prefix mein aur ek kisi doosre prefix mein, aur function ke logs dekho:

echo "hello lambda" > test.txt
aws s3 cp test.txt "s3://$UP_BUCKET/incoming/my report.txt"
aws s3 cp test.txt  s3://$UP_BUCKET/other/ignored.txt     # prefix match nahi → trigger nahi

aws logs tail /aws/lambda/deepdive-s3-logger --since 5m --format short

• Logs mein "key": "incoming/my report.txt" dikha, space ke saath, yaani unquote_plus ne sahi kaam kiya.
• other/ignored.txt ke liye koi log nahi aaya.

Teen aur files upload karo aur logs mein sirf REPORT wali lines dekho:

for i in 1 2 3; do aws s3 cp test.txt s3://$UP_BUCKET/incoming/t$i.txt; done
aws logs tail /aws/lambda/deepdive-s3-logger --since 5m --format short | grep REPORT

Har REPORT line mein Duration, Billed Duration, Memory Size aur Max Memory Used dikhte hain. Kuch lines mein Init Duration bhi hoga, aur wahi cold start hai. Cold start mein AWS ek naya execution environment banata hai, Python runtime load karta hai aur tumhara code import karta hai. Jab environment pehle se garam (warm) hota hai, toh yeh Init wala hissa nahi hota aur function jaldi chalta hai.

• Configuration → General mein Memory ko 128 MB se badha ke 512 MB karo. Lambda mein CPU memory ke hisaab se milta hai, isliye zyada memory dene se function tez chalta hai, aur kabhi-kabhi kam time chalne ki wajah se sasta bhi padta hai.
• Timeout default roop se 3 second hota hai aur zyada se zyada 15 minute ho sakta hai. Isliye Lambda lambe chalne wale kaamon ke liye nahi bana hai.
• Configuration → Environment variables mein LOG_LEVEL=INFO jodo. Code ke andar isko os.environ["LOG_LEVEL"] se padha jaata hai.

• Tumne kam se kam ek REPORT line mein Init Duration dekha aur ek mein nahi dekha.
• Tum bata sakte ho ki cold start kab hota hai aur use kam kaise kiya jaata hai (chhota code package, provisioned concurrency, ya SnapStart).

PL-065 course se — asli console reference

Test event aur timeout/memory config — Lambda ka max timeout 15 min hai.

Neeche wale commands ek chhota Flask app, uski dependencies ki list, ek Dockerfile aur ek .dockerignore file banate hain:

mkdir -p ~/deepdive/app && cd ~/deepdive/app

cat > app.py <<'EOF'
import os
import socket

from flask import Flask, jsonify

app = Flask(__name__)


@app.route("/")
def home():
    return jsonify(
        message="Hello from DEEPDIVE",
        version=os.environ.get("APP_VERSION", "v1"),
        host=socket.gethostname(),
    )


@app.route("/health")
def health():
    return "ok", 200


if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)
EOF

cat > requirements.txt <<'EOF'
flask==3.1.0
gunicorn==23.0.0
EOF

cat > Dockerfile <<'EOF'
FROM python:3.12-slim
WORKDIR /app
# requirements pehle copy: code badle toh pip install wali layer cache se aayegi
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app.py .
EXPOSE 5000
# Production server (gunicorn), Flask ka dev server nahi
CMD ["gunicorn", "--bind", "0.0.0.0:5000", "--workers", "2", "app:app"]
EOF

cat > .dockerignore <<'EOF'
.venv
__pycache__
*.pyc
EOF

• Chaar files ban gayi hain: app.py, requirements.txt, Dockerfile aur .dockerignore.

Apple Silicon wale Mac (M1, M2, M3 ya M4) pe docker build default roop se arm64 image banata hai. Lekin Fargate default roop se x86_64 processor pe chalta hai, isliye arm64 image wala task start hote hi exec format error dekar band ho jaata hai. Isliye build karte waqt --platform linux/amd64 dena zaroori hai. Intel Mac ya Windows pe yeh pehle se default hai, isliye wahan yeh option dene se koi nuksaan nahi.

docker build --platform linux/amd64 -t deepdive-app:v1 .
docker images deepdive-app
docker image inspect deepdive-app:v1 --format '{{.Architecture}}'   # amd64

docker run -d --name dd -p 5000:5000 deepdive-app:v1
curl -s localhost:5000 ; echo
curl -s localhost:5000/health ; echo
docker logs dd
docker exec dd hostname        # container ka hostname == JSON ka "host"
docker rm -f dd

Ab app.py mein ek line badlo (jaise message) aur image dobara build karo. Build ke output mein pip install wale step ke aage CACHED likha dikhega, kyunki dependencies wali layer nahi badli. Dekhne ke baad code ko wapas pehle jaisa kar do.

• curl localhost:5000 JSON response de raha hai.
• Image ka architecture amd64 hai.
• Dobara build karne pe dependencies wali layer CACHED se aayi.

Is step mein tum deepdive-app naam ki ek private registry banaoge, usme login karoge, image ko tag karoge aur push karoge.

Cloud mein container chalane wali service (ECS ya EKS) tumhare laptop se image nahi utha sakti; use image ek registry mein chahiye. ECR AWS ki private Docker registry hai. Iska access IAM se control hota hai, aur yeh push hote hi image ko security vulnerabilities ke liye scan bhi kar sakti hai.

aws ecr create-repository --repository-name deepdive-app \
  --image-scanning-configuration scanOnPush=true

export REPO_URI=$ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/deepdive-app

# Docker ko 12-ghante ka ECR token do
aws ecr get-login-password | docker login --username AWS --password-stdin $ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com

docker tag deepdive-app:v1 $REPO_URI:v1
docker push $REPO_URI:v1

aws ecr describe-images --repository-name deepdive-app \
  --query 'imageDetails[].[imageTags[0],imageSizeInBytes,imagePushedAt]' --output table

• describe-images ke output mein tag v1 dikh raha hai.
• ECR console mein deepdive-app repository ki image kholke uska Vulnerabilities scan result dekh liya.

Is step mein tum deepdive-app-task naam ki task definition banaoge, jo Fargate pe 0.25 vCPU aur 0.5 GB memory ke saath web naam ka container port 5000 pe chalayegi.

Task definition ek recipe hai jo batati hai ki container kaise chalana hai; ise tum docker run ke saare options ka JSON roop samajh sakte ho. Isme do alag roles hote hain, aur interviews mein inke baare mein bahut poocha jaata hai:

• Task execution role ECS ke agent ke liye hota hai. Isi ki permission se agent ECR se image kheenchta hai aur container ke logs CloudWatch mein bhejta hai. Iska aam naam ecsTaskExecutionRole hota hai.
• Task role tumhare app ke code ke liye hota hai. Agar app ko S3 ya DynamoDB jaisi services chahiye, toh unki permission isme di jaati hai. Abhi hamare app ko iski zaroorat nahi hai.

1. ECS console mein Task definitions → Create new task definition kholo aur family ka naam deepdive-app-task rakho.
2. Launch type mein AWS Fargate, OS/Architecture mein Linux/X86_64, CPU mein .25 vCPU aur Memory mein .5 GB chuno.
3. Task role ko khaali (none) chhod do. Task execution role mein Create new role chuno, ya agar ecsTaskExecutionRole pehle se bana hai toh use chun lo.
4. Container section mein naam web rakho (Lab 5 mein yahi naam chahiye hoga), image URI mein <REPO_URI>:v1 daalo, Essential ko yes rehne do, aur port mapping mein 5000 TCP jodo.
5. Environment variables mein APP_VERSION = v1 jodo. Log collection mein Use log collection (CloudWatch) ko on rakho aur Create dabao.

• Task definition ka revision 1 ban gaya hai.
• Task definition ke JSON tab mein executionRoleArn aur awslogs log driver dikh rahe hain.

1. ECS console mein Clusters → Create cluster kholo, naam deepdive-cluster rakho, infrastructure mein sirf AWS Fargate (serverless) chuno aur Create dabao.
2. Cluster ke andar Services → Create kholo. Compute mein launch type FARGATE chuno, task definition family mein deepdive-app-task (latest revision) chuno, service ka naam deepdive-svc rakho aur desired tasks 1 rakho.
3. Networking section mein default VPC aur uske 2–3 subnets rehne do. Security group mein Create new chuno, naam deepdive-ecs-sg rakho aur inbound mein Custom TCP 5000 ko Anywhere se allow karo (yeh sirf lab ke liye hai). Public IP ko ON rakho, kyunki default VPC mein NAT nahi hai aur public IP ke bina task ECR se image hi nahi kheench payega.
4. Load balancing ko none rehne do aur Create dabao. Ek-do minute ruko.
5. Service ke Tasks tab mein task kholo, Configuration section se uska Public IP copy karo aur curl http://PUBLIC_IP:5000 chalao.

• Task RUNNING state mein hai aur curl JSON de raha hai, jisme host field task ka hostname hai.
• Task ke Logs tab mein gunicorn ke start hone wali lines dikh rahi hain.

PL-065 course se — asli console reference

Amazon ECS console landing — is lab mein tum yahi console se cluster/service banaoge.

1. Tasks tab mein task select karo aur Stop dabao. Service ek minute ke andar ek naya task shuru kar degi; yeh tum Events tab mein dekh sakte ho. Dhyan do ki naye task ka public IP alag hoga. Isi wajah se production mein tasks ke aage hamesha ek Load Balancer lagaya jaata hai.
2. Naya version banane ke liye Task definitions mein deepdive-app-task kholo, Create new revision dabao, environment variable ko APP_VERSION = v2 karo aur Create dabao.
3. Service mein Update service dabao, latest revision chuno, Force new deployment pe tick lagao aur Update karo. Deployments tab mein rolling update dekho: pehle naya task healthy hota hai, aur uske baad purana task band hota hai.
4. Naye task ke IP pe curl karo; response mein "version": "v2" dikhna chahiye.

• Band kiye gaye task ki jagah naya task apne aap aa gaya.
• Version v2 deploy hua aur curl ke response mein "version": "v2" dikha.

Pehle eksctl aur kubectl tools install karo, phir cluster banao. Cluster banne mein 15–20 minute lagte hain, kyunki eksctl andar se VPC, control plane aur nodes ke liye CloudFormation stacks banata hai:

brew install eksctl kubectl
# eksctl na mile toh: brew tap weaveworks/tap && brew install weaveworks/tap/eksctl
eksctl version && kubectl version --client

# ~15-20 minute lagte hain (CloudFormation stacks banate hain: VPC, control plane, nodegroup)
eksctl create cluster --name deepdive-eks --region us-east-1 \
  --zones us-east-1a,us-east-1b \
  --nodegroup-name ng-1 --node-type t3.small --nodes 2

kubectl get nodes -o wide

eksctl tumhari ~/.kube/config file ko khud update kar deta hai, isliye kubectl seedha naye cluster se baat karne lagta hai. Nodes ke IAM role mein ECR se image padhne ki permission pehle se hoti hai, isliye tumhari private image bina kisi extra setup ke pull ho jaayegi.

• kubectl get nodes mein do nodes Ready state mein dikh rahe hain.

Kubernetes kuch basic objects se bana hai. Pod sabse chhoti unit hai, jisme ek ya zyada containers hote hain. Deployment batata hai ki kitne pods chalne chahiye aur naya version aane pe unhe dheere-dheere kaise badalna hai. Service pods ke aage ek stable naam aur IP deta hai aur traffic ko pods mein baantta hai. Namespace cluster ke andar ek logical folder hai. RBAC (Role aur RoleBinding) batata hai ki cluster ke andar kaun kya kar sakta hai.

cat > k8s.yaml <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deepdive-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: deepdive
  template:
    metadata:
      labels:
        app: deepdive
    spec:
      containers:
        - name: web
          image: $REPO_URI:v1
          ports:
            - containerPort: 5000
          env:
            - name: APP_VERSION
              value: "k8s-v1"
          resources:
            requests:
              cpu: 100m
              memory: 128Mi
          readinessProbe:
            httpGet:
              path: /health
              port: 5000
---
apiVersion: v1
kind: Service
metadata:
  name: deepdive-app
spec:
  selector:
    app: deepdive
  ports:
    - port: 80
      targetPort: 5000
EOF

kubectl apply -f k8s.yaml
kubectl get deploy,pods,svc -o wide
kubectl port-forward svc/deepdive-app 8080:80
# doosre terminal mein:  curl localhost:8080   (baar baar — host badalta dikhega)

• Do pods Running state mein hain aur READY column mein 1/1 dikh raha hai, yaani readiness probe pass ho gaya.
• port-forward ke through curl JSON de raha hai jisme version: k8s-v1 hai.

Ab Kubernetes ki chaar khoobiyan ek-ek karke dekho: pods ko maarne pe naye pods ka aana, replicas badhana, naya version rollout karna aur rollout ko wapas lena:

kubectl delete pod -l app=deepdive --wait=false && kubectl get pods -w   # naye pods turant (Ctrl+C)
kubectl scale deployment deepdive-app --replicas=4 && kubectl get pods
kubectl set env deployment/deepdive-app APP_VERSION=k8s-v2
kubectl rollout status deployment/deepdive-app
kubectl rollout history deployment/deepdive-app
kubectl rollout undo deployment/deepdive-app
kubectl describe deployment deepdive-app | head -30

• Pods delete karne pe naye pods aa gaye, scale karne pe pods ki ginti 4 ho gayi, aur rollout aur undo dono chal gaye.

Kaam khatam hote hi cluster delete karo:

kubectl delete -f k8s.yaml
eksctl delete cluster --name deepdive-eks --region us-east-1

• CloudFormation console mein eksctl-deepdive-eks-* naam ke saare stacks DELETE_COMPLETE dikha rahe hain.

1. IAM mein Roles → Create role kholo, AWS service mein EC2 chuno, policy AWSElasticBeanstalkWebTier jodo, role ka naam aws-elasticbeanstalk-ec2-role rakho aur Create dabao. EB console ab yeh role khud nahi banata, isliye isko pehle banana padta hai.

Phir app ka zip bundle banao:

cd ~/deepdive/app
echo "web: gunicorn --bind :8000 --workers 2 app:app" > Procfile
zip -r ~/deepdive/lab2/eb-app-v1.zip app.py requirements.txt Procfile
unzip -l ~/deepdive/lab2/eb-app-v1.zip

Procfile Elastic Beanstalk ko batata hai ki app kaise start karna hai. EB ke Python platform pe nginx port 80 pe requests sunta hai aur unhe port 8000 pe tumhare app ko aage bhejta hai, isliye gunicorn ko :8000 pe bind kiya gaya hai. Ek aur zaroori baat: zip ke root mein hi files honi chahiye, kisi folder ke andar nahi. EB deploy fail hone ki yeh sabse common wajah hai.

• Zip ke root mein app.py, requirements.txt aur Procfile hain.

1. Elastic Beanstalk console mein Create application dabao. Environment tier mein Web server chuno, application ka naam deepdive-eb aur environment ka naam deepdive-eb-dev rakho.
2. Platform mein Python (latest Amazon Linux 2023 branch) chuno. Application code mein Upload your code chuno, version label v1 rakho aur eb-app-v1.zip upload karo.
3. Presets mein Single instance (free tier eligible) chuno aur Next dabao.
4. Service role mein Create and use new service role chuno aur EC2 instance profile mein aws-elasticbeanstalk-ec2-role chuno. Key pair ko skip kar do. Networking mein default VPC aur public IP on rakho, aage ke pages pe Next dabate raho aur ant mein Submit karo. Environment banne mein lagbhag 5 minute lagte hain.
5. Environment ka Domain URL kholo; wahan tumhara JSON response aana chahiye.

Ab yeh dekho ki EB ne tumhare liye kya-kya banaya. CloudFormation mein awseb-e-...-stack naam ka stack kholo aur uska Resources tab dekho. EC2 console mein ek instance aur ek security group dikhega. S3 mein elasticbeanstalk-us-east-1-<account> naam ki bucket mein tumhara zip file rakha hoga.

• EB ke URL pe "message": "Hello from DEEPDIVE" aaya aur environment ka Health Ok hai.
• Tumne CloudFormation mein EB ka stack aur uske resources dekh liye.

1. Environment mein Configuration → Updates, monitoring, and logging kholo aur Environment properties mein APP_VERSION = eb-v2 jodo, phir Apply dabao. Events tab mein update hota dekho, aur uske baad URL pe "version": "eb-v2" dikhna chahiye.
2. app.py mein message badlo, naya zip eb-app-v2.zip banao, Upload and deploy dabao aur version v2 rakho. Application versions page pe ab dono versions dikhenge; v1 select karke Deploy dabane se ek click mein rollback ho jaata hai.
3. Logs → Request logs → Last 100 lines kholo; wahan nginx aur web.stdout, dono ke logs dikhenge.

• Environment property se version badla, v2 deploy hua, aur tumne v1 pe rollback karke dekha.
• Tumne request logs dekh liye.

Pehle default VPC ke do subnets aur uska default security group nikaalo, phir unse ek Fargate Spot compute environment aur ek job queue banao:

cd ~/deepdive/lab2
VPC_ID=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true --query 'Vpcs[0].VpcId' --output text)
SUBNETS=$(aws ec2 describe-subnets --filters Name=vpc-id,Values=$VPC_ID Name=default-for-az,Values=true \
  --query 'Subnets[0:2].SubnetId' --output json)
DEF_SG=$(aws ec2 describe-security-groups --filters Name=vpc-id,Values=$VPC_ID Name=group-name,Values=default \
  --query 'SecurityGroups[0].GroupId' --output text)

cat > ce.json <<EOF
{
  "computeEnvironmentName": "deepdive-fargate-spot-ce",
  "type": "MANAGED",
  "computeResources": {
    "type": "FARGATE_SPOT",
    "maxvCpus": 4,
    "subnets": $SUBNETS,
    "securityGroupIds": ["$DEF_SG"]
  }
}
EOF
aws batch create-compute-environment --cli-input-json file://ce.json
sleep 20
aws batch describe-compute-environments --compute-environments deepdive-fargate-spot-ce \
  --query 'computeEnvironments[0].[status,state]'

aws batch create-job-queue --job-queue-name deepdive-queue --priority 1 \
  --compute-environment-order order=1,computeEnvironment=deepdive-fargate-spot-ce

• Compute environment VALID aur ENABLED hai, aur queue VALID hai.

Ab ek job definition register karo jo ek chhota container chalati hai, aur phir ek array job submit karo jo us container ki paanch copies ek saath chalata hai:

EXEC_ROLE=$(aws iam get-role --role-name ecsTaskExecutionRole --query Role.Arn --output text)
# Na mile toh: ECS console mein task definition banate waqt "Create new role" se banta hai (Part F2)

cat > jobdef.json <<EOF
{
  "jobDefinitionName": "deepdive-hello",
  "type": "container",
  "platformCapabilities": ["FARGATE"],
  "retryStrategy": { "attempts": 2 },
  "containerProperties": {
    "image": "public.ecr.aws/amazonlinux/amazonlinux:2023",
    "command": ["sh", "-c", "echo Job index \$AWS_BATCH_JOB_ARRAY_INDEX on \$(hostname); sleep 20; echo done"],
    "resourceRequirements": [
      { "type": "VCPU", "value": "0.25" },
      { "type": "MEMORY", "value": "512" }
    ],
    "executionRoleArn": "$EXEC_ROLE",
    "networkConfiguration": { "assignPublicIp": "ENABLED" }
  }
}
EOF
aws batch register-job-definition --cli-input-json file://jobdef.json

JOB_ID=$(aws batch submit-job --job-name hello-array --job-queue deepdive-queue \
  --job-definition deepdive-hello --array-properties size=5 --query jobId --output text)

# SUBMITTED → RUNNABLE → STARTING → RUNNING → SUCCEEDED  (2-4 min)
aws batch describe-jobs --jobs $JOB_ID --query 'jobs[0].[status,arrayProperties.statusSummary]'
aws batch list-jobs --array-job-id $JOB_ID --query 'jobSummaryList[].[arrayProperties.index,status]' --output table

Logs dekhne ke liye CloudWatch mein Log groups mein /aws/batch/job kholo. Har child job ki alag log stream mein "Job index 0" se "Job index 4" tak ki line dikhegi. Dhyan do ki heredoc mein \$AWS_BATCH_JOB_ARRAY_INDEX ke aage backslash lagaya gaya hai. Aisa isliye kiya gaya hai taaki yeh variable tumhare laptop pe nahi, balki container ke andar expand ho, jahan uski asli value hoti hai.

• Array job ke paanchon child jobs SUCCEEDED ho gaye.
• Logs mein index 0 se 4 tak alag-alag lines dikh rahi hain.