I suspect my alternator is going bad but I want to be sure before I replace it.

My car: Honda civic 2014 LX, 160,000 miles, automatic transmission 

I recently replaced my battery as well. Serpentine belt changed at 110,000 miles. 

The problem I face is that some mornings when I crank the car, it turns the car but it feels like its struggling to power the car, like low battery. 

Since my car battery is new and I had my starter replaced by honda dealership at 110,000 miles. I suspect it could be alternator.

I got a multimeter to check the battery and alternator but I am not sure if I am doing it right.

In the morning the battery showed 12.78v and when I turned on the car it went to 14.40v and when I turned on the AC, lights and radio, when the fan turned on the voltage would go to 13.60 and back to 14.20v or 14.40v. The only time it went low was when I tried to roll down window then it would drop to 13v to 12.8v but jump back up to 14.4v with AC and radio and lights running.

I did same test the night before, after driving the car whole day and I got different results.

Battery was 12.68v when car was off, when I  turned on the car it would go to max 12.80v to 12.90v but when I turned on the lights, AC and radio then it jumped to 14v but dropped to 13.20v and stayed in that area and when car fan started then it would go to 12.80v then back to 13.20v.

Am I testing the car at the right conditions? Morning or after driving whole day? Is this voltage inconsistency a symptom of bad alternator?

Note: I am still learning, I do basic brake pad and oil change at home,  I don't have advanced skills