If you do use lighter fluid to get your fire going apply it sparingly and evenly. You want each and every coal to get a dose of lighter fluid, but not too much. You definitely don't want a pool of it forming in the bottom of your grill.
Secondly let the lighter fluid soak in for 30 seconds to a minute. This allows the fumes to escape and lets the lighter fluid sink in. When you do put a match to the coals you won't get a big roaring fire, but the fire will spread over the coals. It actually only takes a couple of minutes of fire to heat a charcoal coal to the point of combustion.
A charcoal fire is ready to cook with when the coals are hot and white, but also when the smoke coming off the fire is white and at a minimum. A smoky charcoal fire is not good for grilling.