Physical Addresses
...is when every address "visible" to a program directly relates to an address on the address bus between CPU and RAM. If I access 0x0023 5fc4, I mean the address identified by the bit pattern 0x0023 5fc4 on the address bus.
In this memory model, every executable or library must either use PIC (position-independent code), or come with relocation tables so jump and branch targets can be adjusted by the loader.
The AmigaOS used this memory model, in absence of a MMU in early 680x0 CPUs. It is most efficient, but it does not allow for protecting processes from each other, thus it is considered obsolete in today's desktop operating systems. It is also prone to memory fragmentation; certain embedded systems still use it, though.
Virtual Addresses
The advent of MMUs (Memory Management Units) allowed to play tricks to the addressing. A virtual address could be mapped to any physical address. It was possible to provide each executable with its own address space, so that memory always starts at 0x0000 0000. This relieves the executable loader of some relocation work, and solves the memory fragmentation problem - you no longer need physically continuous blocks of memory. And since the kernel is in control of the virtual-to-physical mapping, processes cannot access each other's memory unless allowed to do so by the kernel.
Paging
Having an individual virtual-to-physical mapping for each address is of course ineffective. The traditional approach to virtual memory is to split up the available physical memory into chunks (pages), and to map virtual to physical addresses page-wise. This task is largely handled by the MMU, so the performance impact is low, and generally accepted as an appropriate price to pay for memory protection.
physical memory virtual memory (A) virtual memory (B)
+-------+ +-------+ +-------+
00x |H E L L| page table |H E L L| page table |H A V E|
01x |R L D !| for proc. A |O W O| for proc. B | L O T|
02x |O W O| 00x => 00 |R L D !| 00x => 03 |S O F|
03x |H A V E| 01x => 02 |#######| 01x => 05 | F U N|
04x | F U N| 02x => 01 |#######| 02x => 06 |#######|
05x | L O T| 03x n.a. |; - ) | 03x => 04 |; - ) |
06x |S O F| 04x n.a. +-------+ 04x => n.a. +-------+
07x |; - ) | 05x => 07 05x => 07
+-------+
paging illustrated: two process with different views of the
same physical memory
Memory Protection
Probably the most useful application of virtual addresses is memory protection. In a memory-protected environment, every process (executable) gets its own address space. To the executable, it looks like it is running alone on the system, no-one else there but the kernel. The bargain is that a misbehaved, malicious, or buggy executable cannot damage / corrupt other processes in the system. If the executable fails, the rest of the system keeps running instead of meeting the Guru Meditation, Blue Screen of Death, or whatever your system tells you when the system gets borked.
Virtual Memory (Swap file/partitions)
The next step is, instead of reporting an "out of memory" once the physical memory runs out, is to take pages that are not actually accessed currently, and write them to hard disk (swapfile or -partition) - freeing up the physical memory page. This is referred to as "paging out" memory.
This requires additional bookkeeping and scheduling, introduces a severe performance hit when a process accesses a page that's currently swapped out and must be swapped in again from hard drive, and requires some smart design to run efficiently at all. Do it wrong, and this one part of your OS can severely impact your performance.
On the other hand, your "virtual address space" grows to whatever your CPU and hard drive can handle. In concept, CPU caches and RAM simply become cache layers on top of your hard drive, which represents your "real" memory limitation.
Page swapping systems relies on the assumption that, at a given time, a process does not need all of its memory to work properly, but only a subset of it (like, if you're copying a book, you certainly don't need the whole book and a full set of blank pages: the current chapter and a bunch of blank page can be enough if someone can bring you new blank pages and archive the pages you've just written when you come short on blank pages, or bring you the next chapter when you're almost done with the current one). This is known as the working set abstraction. In order to run correctly, a process requires at least its working set of physical pages: if less pages are provided to the process, there's a high risk of thrashing, which means the process will be constantly requiring pages to be swapped in -- which forces other pages from this process's working set to be swapped out while they should have remained present.
Note: there are alternatives to page-swapping like segments-swapping and process-swapping. In those cases, the swap is rather user-software controlled, which puts more stress on the application developer and leads to longer swapping burst as the logical things to be swapped are bigger than 4K pages.
Other note: mainstream desktop OSes have a speculative algorithm that tries to reduce the 'page miss' frequency by loading more than what is required, and hoping that these extra pages will be useful. As programs tend to have localized access and that disks can read a track of N sectors faster than N independent sector, speculative swap-in may pay.
Summary
Virtual addresses and paging are A Good Thing (tm), even a requirement for every decent OS not necessarily limited to desktop/server systems (sometimes even embedded ones). Virtual memory certainly is a nice-to-have (and every modern desktop OS has it), but it comes at a cost and requires significant additional design and know-how to get right.
paging