What exactly is this bit-story?

Computers are digitial. Digital means we build components from"analog" hardware, like transistors, resistors, etc., which work with two states: On and Off. Usually we can do this by employing reversed Z-Diodes, which are not conductive until a certain voltage threshold is reached. That way, you either have 0V or <threshold>+ Volt. The resulting components are called "logical gates". Some examples for gates include "AND", "OR", "NAND", "XOR", etc. With these gates, your computer hardware can be built easily in a way which allows for storing states (e.g. NAND flip-flops) and do logical things, like mathematic computations. One storage-place can save the state of either 0 or 1. This state is called "bit". Bits work in the binary system. Let me translate that for you to our widely used decimal system:

Binary	Decimal
0	0
1	1
10	2
100	4
101	5
1010	10

As you can see, you need 4 bits to save the decimal number 10, so we need multiple bits to represent one number in a different mathematical systems. 64bit means you can saves binary numbers with a length of 64 states; or logical values where different bits have special meanings.

What exactly is 64-bit software

64bit software in general just means that your software uses memory addresses which are 64bit long and data types (integer, word,...) which are 64bit wide. You can read more on Wikipedia.

Whats the difference between 32-bit and 64-bit software?

The biggest difference is that 64bit software can use a lot more memory, since it can use bigger pointers, hence a bigger address space. You could emulate this by combining two 32bit data fields (which was done in practice), but with the introduction of native 64bit capabilities in hardware, we can take the easy route :)

The main advantage of 64bit software, compared to 32bit software, really just comes from this easy fact. Since we have more memory space, we can store more stuff there so it can be access faster than from disk. The result: Whenever you need more than 4GB RAM (that's the most 32bit can address) you really benefit from 64bit architecture. Your operating system is one of the software pieces which benefits the most from the new architecture, since it has to manage resources for many programs which most likely will exceed 4GB.

What are the benefits of writing 64-bit software on 64-bit systems instead of just writing 32-bit software for both systems?

Just as I said, the OS will benefit a lot from 64bit software, so most people today have 64bit systems. Developers at some point stopped caring for memory optimizations (often in favor of a more readable code and easier logic), so you will exceed 4GB RAM easily with just a modern Win10 and Office2016. The problem is that 32bit software has to run in a kind of emulation on 64bit systems, which makes them slower compared to 64bit software on a 64bit system. In most situations, however, the difference doesn't really matter.

Personally, I would write performance software (for example the game I am working on) for 64bit and other stuff for 32bit and maybe offer optional 64bit binaries, if I have the time and motivation. Alternatively you can just go opensource and let people compile for whatever architecture they want. That's one of the things I really like about source-based linux distributions, like Gentoo and Stali. I can decide and compile stuff myself :)