Clarifications on AWS auto-scaling lifecycle and cool down time
I have been confused by AWS auto-scaling things for a long time. AWS auto-scaling is complicated, while its document is also organized “complicated”, I had read its document many times but still wasn’t able to get all things clear.
Today I have to pick up something related with AWS auto-scaling so I try to figure out the mess of AWS auto-scaling again, fortunately, this time I think I’m clear about it.
Confusions
When I think back I find myself actually having been confused by such points, so I just want to stress these points in this article.
- What on earth the
inService
state means in scaling out lifecycle? Does it mean the instance or our application process having finished starting? - If already exists new lauched instances due to previous scaling out which are entering the
inService
state, will a new fired alarm causes another scaling out? - In the simple scaling policy, cooldown time starts at the time the instance finishes launching or enters the
inService
state?
Clarifications
What on earth the inService
state means in scaling out lifecycle?
The answer for the first confusion hides in this sentence of the AWS doc:
When each instance is fully configured and passes the Amazon EC2 health checks, it is attached to the Auto Scaling group and it enters the
InService
state.
I have highlighted the condition the instance enters into inService
state, but If you click the link to find out this sentence, you will find it's really an un-noticable sentence, whilst I think it should have been written to more noticable, as it's about the state transition.
To be clearer, passing Amazon EC2 health checks means the instance not just have been started, but must have finished operating system starting, configuring and the process responsible for responding EC2 health checks has also run up. The process responsible for responding EC2 health checks, however, normally means our own application process. So, normally, when an instance enters the inService
state, it means the instance is fully ready to serve client requests.
If already exists new lauched instances due to previous scaling out which are entering theinService
state, will a new fired alarm causes another scaling out?
In common sense, the answer should be “No”, while aws document doesn’t stress this point, only in the “Differences between step scaling policy and simple scaling policy” section it mentions:
The main issue with simple scaling is that after a scaling activity is started, the policy must wait for the scaling activity or health check replacement to complete and the cooldown period to expire before responding to additional alarms.
https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-scaling-simple-step.html#SimpleScaling
In the simple scaling policy, cooldown time starts at the time the instance finishes launching or enters the inService
state?
I believe I’m not the only one who has question on this. It can be more confusing if you don’t have a clear awareness of the second question. There’s a saying from aws that:
When a lifecycle action occurs, and an instance enters the wait state, scaling activities due to simple scaling policies are paused. When the instance enters the
InService
state, the cooldown period starts.https://docs.aws.amazon.com/autoscaling/ec2/userguide/lifecycle-hooks.html#lifecycle-hook-considerations
It says that the cooldown starts when the instance enters the inService
state, honestly this confused me at first, if the cooldown starts only when the instance enters the inService
state, then what happens when the instance hasn't entered this state? Does it mean before the instance enters the inService
state, a another new scaling activity can be fired?
I have been in confusion until I realize the answer of the second question, i.e. before the newly launched instances get into inService
state, the following scaling activity is suppressed, this for sure. Only when I realize this the above saying makes sense to me now, while the first part the that sentence "When a lifecycle action occurs, and an instance enters the wait state, scaling activities due to simple scaling policies are paused." does not clearly state this point.
Ok now we are certain that cooldown starts after the inService
state, but why?
Mostly because the we need more time for the effects of the newly added instance to be reflected. e.g. AWS alarm metric is aggrated every 5 minutes, when the newly added instance is in service, it still need 5 minutes before the CPU usage decrease can be observed[1].