Exhaust back pressure is required on every internal combustion vehicle. 

Without getting into the technical jargon, when you delete the cats you speed up the exhaust flow. Catalytic converters are designed to burn off pollutants as the exhaust fumes move through the exhaust system. Manufacturers design their exhausts around a certain diameter exhaust pipe, usually 2.5". Many, many hours of R&D go into figuring out exactly what the maximum flow through the pipe should be in order to provide the exact pressure needed to generate maximum torque and power.

The reason catback systems are so expensive is that they are not only made of better materials than stock, but they also typically come with free-flowing mufflers and sometimes larger pipe diameter. These exhaust companies spend millions doing the same thing the manufacturer did: a lot of math to determine the best setup and sound. Exhaust systems are not just bent pipes as many think...there is a science to it.

If you purchase a catback designed for your car, you will get better flow characteristics as well as gain torque vs losing it by deleting the cats. 

It should be noted that it has been scientifically proven that deleting catalytic converters does not add any power, in many cases will cause loss of power, and makes the car run worse than stock.

I think your experiment clearly shows that it does matter. Some exhaust velocity is needed to scavenge spent exhaust gases from the engine cylinder, so a cat back system is usually recommended. I'm sure @mod_man can break it down better for you.