When the mortar on your brick is breaking up a bit, or scaling off, you cannot simply put a skim coat of new mortar over the old, it will not hold. If someone has done that before on your house you know it doesn't hold, that is why you are redoing the job now.
Mortar sticks to the brick, not to the old mortar. Hence you have to remove old mortar as far back as the mortar line is wide. This will assure that the new mortar will stick to the two bricks and stay put.
The best, but not the fastest way to remove some of the old mortar is to use a mortar chisel as in the first photo. This special chisel comes in a variety of sizes so get the one that fits the brick spacing on your wall. A masonry grinder is commonly used for removing mortar but as you can see in the second photo, it can easily cut into the brick, damaging the appearance of the wall. If for some reason you need to completely remove a brick it is best to drill several holes right through the mortar with a masonry bit. This will make it easier for the chisel to remove the mortal all the way through to the other side of the brick.
Follow this link to get details about preparing the mortar for repointing.
To get the mortar into the crack, the best method is to use a special thin spatula to scrape the mortar off of the trowel directly into the slot as in the fourth photo above. Pushing in tightly enough to ensure that the mortar makes good contact with both the top and the bottom brick is essential for a watertight joint. Scrape off any excess as you go to avoid staining.
When the mortar sets enough to take a thumb print, you should go back with a rounded trowel, or whatever matches what was previously done on your wall, to compact the face of the joint. This step is what really seals the face of the mortar against water penetration.