Yep.That's the right edge of the drive performance table kicking in.
You can either have ships that are big, or ships that are fast, but not both without high tech.
I have some pretty strong internal limits for ship design. J3 and M4 are pretty much the upper limits of my designs.
Honestly I often ponder how to fit in M 0.5 ships in especially for larger freighters.