Firstly, you want to check the influent valving before the pump. The pool janitor recommends that the skimmer pull in about 75% of the total flow into the pump. For example, if your pool has two influent valves, a main drain and a skimmer, close the main drain halfway while leaving the skimmer valve full open. If your pool has an attached spa, crack the spa drain valve open only very slightly, or leave it closed altogether.
If you haven't purchased a leaf rake, or a "drag bag", as I sometimes call them, and are holding on to that flat "dip & flip" net that your builder gave you; you are creating your own hell. I strongly encourage the purchase of a nice leaf rake. There are also chemical products which are used to keep surface tension high, moving small debris to the sides of the pool. Another possible problem could be the condition of the weir in the skimmer; you know, that flapper gate thing. Make sure it is operating properly so that it creates a draw or "waterfall" into the skimmer basket. Also, check that the water level is not so high that it is above the opening of the skimmer.
Lastly, you may need to trim some of those trees and bushes near the pool. My pool, for example, was specifically built with no vegetation anywhere within wind shot.
