If the price is out of equation, get Avalon. For the best bang for your buck, get the Camry. 

I would go with the Camry.  2006 was the last year of that generation (model years 2002-2006) and one of the best.  Also, the engine you are looking at is the 2.5L I4 and for longevity, repair, and maintenance costs the 4 cylinder is better than the 6 cylinder (which the Avalon has).  As for the Avalon, 2006 was the first model year of that generation and it had issues with the biggest one being sudden loss of engine power.  There was faulty oil line which could leak while driving and if ignored result in engine failure (worst-case).

https://carkiller.com/scottykilmer/postid/106360/

https://www.carcomplaints.com/Toyota/Camry/2006/

https://repairpal.com/massive-oil-leak-and-loss-of-oil-pressure-150

https://www.carcomplaints.com/Toyota/Avalon/

http://www.toyotaproblems.com/models/avalon/generations/3/

https://www.toyotanation.com/threads/prevalent-issues-3rd-gen-avalon.1441730/

One word of caution:  the 2.5L I4 engine in the Camry is the infamous 2AZ-FE which had oil burning but that issue did not occur until the next generation of Camry (first model year being 2007) and also effected engines built in the US.  The 2006 Camry does not seem to have been impacted.  However, I would make sure the previous owner has a documented service history especially religiously changing the engine oil on time (no more than every 5,000 miles) to mitigate issues due to lack of maintenance.