I am the original owner of a 2011 Toyota Sienna LE 2GR-FE engine 2WD with approximately 140k miles, only driven in CA (hot summer, almost never under freezing conditions).  Transmission oil drain/filled with Toyota ATF using proper temperature measurements, at 100k and recently at about 138k because I thought it couldn't hurt to replace the ATF to see if would solve my problem (it didn't obviously).  Brake fluid has been changed and is clear, and at proper level but the master brake cylinder is the original - no problems with the brakes that I can tell.

The car drives well and the transmission shifts like a dream from gears 2 and up.  However when going from stop to letting go of the brake quickly there is a momentary clunk sensation (no sound, just feel the slight lurch forward) when engaging into first gear.  If I let go of the break slowly this lurch forward/clunk sensation does not occur.   This is repeatable, again with how quickly the foot is taken off the break (even without pressing the accelerator pedal).  I think this "clunk"/lurch also occurs when I go into reverse (but maybe I'm imagining things) also.  

Is this an issue with the transmission or possibly the brakes?  It's almost as if the car is forgetting how to slowly drop into gear from stop to 1st without lurching if you let go of the brake.  The rest of the gear shifts very well without any slippage.  Any ideas what the next step should be to troubleshoot this?

Would much appreciate your help here.