Yes this is correct. Wasm-native Stable Memory has recently been approved so this allows us to avoid the API overhead in many cases, but stable memory is still not as fast as the main memory.
One reason is that stable memory uses a 64-bit address space which means we need to insert bounds checks around every access. A second difference is that we need to track which pages have been accessed and limit the total number of accessed pages (if a canister could touch all of stable memory during an execution, then a few canisters running in parallel could use up the entire replica’s memory).
These checks can both be skipped on regular memory (because the address space is 32-bits and the total size is only 4 GiB), but if we used the Wasm memory64 proposal to support a larger main memory then we would need to add all these checks for regular memory operations.