I read an interesting article on creating deadlines. And the idea was that you need to come up with 3 times:

1. Worst case
2. Best case
3. Most likely/Expected case.

This way you have a range to work within for yourself.

These estimates are based on your experience and how complicated the task is and then add 10/20%. The article I read suggested to double the timeframe, based on statistical analysis if you double the timeframe, and are new to estimations, then you'll be something like 80% likely to actually hit the deadline. It follows the standard deviation principals in statistical maths, bell curve type stuff.

Personally, in previous jobs I used to be really terrible at estimations even though I knew the field. In my current role I seem to be estimating correctly. Although recently I have missed a deadline by a couple of weeks, due to unforeseen bugs/uncovered more underlying issues. This is why we add the safety factor of at least 10/20% of the timeframe estimated.

An important thing to remember is that things will always change and if you can estimate correctly don't get over confident in your abilities, just correctly analyse. It will get easier with time, but the most important thing to remember is to always communicate. E.g, if you have a deadline due in 1 month then let your team know your progress each week or when there are complications that you need additional help with the work or if anything changes.

This communication allows the rest of the team, as well as yourself, to adapt schedules and workloads in a pro-active/preventative nature instead of reactive one.

Hope this helped :)

Update:

My team use gitlab, so I try and get everyone using the milestones/issue tracking etc on there, coupled with the company quarterly goals to streamline task/project management. It really helps. People, who aren't tech for example, can go on gitlab and see the progress of a project against a milestone to see % complete and any complications etc + skype, although gitlab did recently acquire gitter so we should see some chat integration/similar to slack in there in the future :)

The best estimates I have been able to give are by splitting the work into as many smaller tasks as possible. They should be small enough that you are very confident of the estimated time. A day's work is good for many people (ie, if any task is more than a day's work, split it). For some others, it's half-a-day.

If you do have really very uncertain things, give an estimate for how you can get more clarity, and then do the real estimate only once you get reasonable clarity. Alternatively, make an initial estimate (but make it explicit that it is rough), and re-estimate when you have more clarity. It's important to get the uncertain things out of the way first before doing real work.

For information, I'm a freelance web developer.

I write out on a piece or several pieces of paper all the steps I'm going to need to complete a project, which is essentially the same as breaking it down into smaller tasks as others have mentioned. This also means you can query potential problems before you start plus you have a guide for when you do start. The smaller tasks are then estimated based on previous experience or best guess. I then need to factor in things such as, deploying to a test environment (and setting this up), as well as deploying to live (again, setting this up or whatever is required), debugging and testing time, plus time to allow for unforeseen problems.

You'll never be accurate all the time but you should be somewhere close.

You estimate it by using past experience. For example, If I'm to tell you that I need a login process implemented, how would you estimate it? Because a login can be a lot of things, not just two fields that you need to validate against a database.

First of all, you need to learn to ask questions. All kinds of questions in order to know what you're supposed to do. In this example above, you should ask me more about the login process itself - how the user is supposed to log in, what parameters is he going to use, what should happen if his account is disabled, should we allow log in using other accounts (like Google or Facebook). So - learn to ask as many question as you can.

Now when you know everything (or think you know everything), what you need to always have in mind is that 99% of the people do not know what they want and that is important because that is also a part of equation. With that in mind, I always tell you the worst case scenario in days (where 1 day = 8h) accompanied with a sentence "20% more or less" which should give you enough room to really finish it even if you screwed up with your estimate.

The third thing you need to know is that estimates change so it is not a big deal to tell a client that you are going to spend more time than you initially thought. But do not do it 24h before deadline, do it before you've spent 50% of the estimated time. If you communicate with your client, even if you miss a deadline and your original estimate, people won't mind. If you lie to them and then try to find some stupid reason why you're late (like - you had to fetch a medicine for your sick aunt) then you're done.

The fourth thing is to count with change requests that will come in the middle of your work. That tends to screw up everything, so you need to learn to fight it. How? By asking questions (that was the first thing) and by making a specs. Write everything down, send it in mail (not Skype, Viber, Slack, iMessage or w/e you use for fast communication) and ask your client to approve it and sign it off as a final version. If he comes later and says he wants to add something, you should:

1. Shrug and say ok, but that will prolong my implementation, hence my estimate is going to be wrong,
2. Estimate a CR itself, then tell him you are going to do it but it is a low priority.

And that is about it :)