I just worry if there is any unavailable period (or break-time) when it's scaling?
Technically, at the Fargate level, the answer is no as long as you have your service set with a minimum task count >= 1.
The wiggle room in saying "no" though, is if your app spikes from 70% CPUUtilization to 100%, the application itself may become unresponsive before Cloudwatch is able to trigger an alarm which in turn triggers the service to scale out.
although before that time, it's quite high
Keep in mind, scaling actions aren't instantaneous. If you're using a Cloudwatch metric for CPUUtilization with a period of 60 seconds, and a threshold of 2, that means your task would have to be > 80% utilization for over 2 minutes before autoscaling was triggered.
In addition to this, Fargate startup time is slower than ECS launch type startup time, because AWS has to do some magic behind the scenes - specifically download the image and also attach an ENI - to make it "serverless".
So, if your application goes over 80% utilization, you wouldn't see it autoscale immediately. This might explain what you're seeing in Cloudwatch with the utilization being high enough to trigger scaling, dropping off to 2% but after the scaling has already been triggered.