How do you design a program to run fast?
The first question you need to ask is what kind of I/O this program will use? Networked I/O’s, database disk I/O’s or accessing data only in memory? In principle, you would want to minimize the number of networked and disk I/O’s as much as possible because this is usually the real bottleneck of a program’s performance. Once this is certain, your program will then structure in a way to minimize I/O operations, which also means half of the ‘design’ problem is already solved.
The second question, then, is to ask if your program has synchronozation and locking contention, that will also cause your program to halt. If you can avoid that, about one-fourth of the design problem is solved.
The third question, then, is if the program has been overdesigned or having excessivg logging. Keep things simple and in clean structure.
The fourth question, then, maybe comes later than expected, is to ask what data structures and algorithms to use to even further minimize the tasks a program needs to do, given the condition that all the above quesitons has being addressed properly.
After all the above, you then can start thinking about memory management, for example, avoid memory leaks or using memory pool to avoid page faults, or doining data caching when necessary etc.
One thing worth mentioning is that, even though the data that a program accesses is solely in memory, it can still be further optimized (reducing memory access) by declaring your data structure in a way that many data can already be cached in L0/L1/L2 cache. This is called Data-Oriented Programming.