language. Good micro-level code is only one component of good performance, and it is usually not the most important. Choosing the right data structures and algorithms is the most important aspect of getting good performance. Having the fastest linked list code in the world is a waste if you should have used a hash table. Similarly, it isn't smart to use a hash table when all you ever do is sequential access. It is also misspent energy to worry about whether a function call should be inline when the function does a disk operation.
Remember that there is no code faster than no code--the best way to speed up code is to eliminate it. If you can avoid a computation by caching the result, or by finding some other way to get what you need, that's a big win.
There's a lot of oral tradition in this area (except that it is written down), and it's well worth reading everything you can get your hands on. Here are some recommended written sources of information on this topic:
- Writing Efficient Programs (Bentley).
- "Hints for Computer System Design" (Lampson) in Proceedings of the Ninth ACM Symposium on Operating System Principles, Operating System Review Vol. 17 No. 5 (pp. 33-48). Good observations by someone who's been through the wringer a few times.
- Programming Pearls and More Programming Pearls (Bentley).
- The Psychology of Computer Programming (Weinberg) has some choice bits about what drives programmers to worry about efficiency.