There are programming fields i the gray area between programming and design work where math is not very important. There are also some business logic areas where all you need to know is basic logic and data types, so not very much in the way of math. What you cannot do without math is develop things like game physics engines, large-scale applications where complexity analysis is needed, and just about anything robotics related except for the mmore trivial things.
Those that are very good at math generally consider jobs that do not involve math to be boring, so they won't compete for them -- but on the other hand, there are more people by definition competing for jobs that only require average math skills.
You should still try to improve your math, especially since improving math improves your thinking, but concentrating on logic and understanding data types and representations (for example, knowing how floating point numbers work inside) can get you a long way.