Imply is to implicate or suggest or hint at something without expressing it plainly or directly, which is something done by the speaker, the writer, or someone performing an action.
Infer is to interpret or conclude or deduce from the evidence that is presented, which is something done by the listener, the reader, or the person who sees the action being done.
- When the company president said that he was going through tough financial times, we inferred that he meant there would be no raises this year. (We are the listeners; we made a decision about what we assumed he was hinting at.)
- The company president was implying that there would be no raises this year when he showed us the graph of declining profits. (The company president is the speaker; he hinted that there would be no raises.)
You could even say We inferred that the company president was implying there would be no raises this year when he said he was going through tough financial times.