Yes, numbers is a lot of it, but the realism slows down a combat because (realistically) you have a lot of decisions to make (parry, form of attack, drop one weapon and draw another, run away, etc. - and those are just the high-level decisions). Each one of those decisions is made in a split-second - and rather instinctually - IRL. If you aren't well-trained to make that split-second decision, then you often die just after that split-second.
Players not only have to make those decisions, they have to verbalize them to the ref, the ref has to know (or look up
) which roll/table to use for that decision, a roll has to be made, conclusion reached.... Then you have to do the same thing with the character with the second initiative....
So, the more realistic you make things (even if its just tasks, not combat), the more decisions you allow. Or, the more layers of definition you use (penetration and critical rolls, i.e.). Etc....
That's the big reason for "simplifying" combat and tasks - simulating real life takes loads of time.