The first thing to note is that like many countries in Europe, receiving calls is free.  The calling party pays for both ends of the connections.

Before you start worrying about this, you need to ask yourself "Do I really need land-line service?".  The reason why is that it is very expensive to set up.  You have to buy a "subscription right", usually from NTT, for what amounts to hundreds of dollars/euros or more.  Aside from that, obviously if you move, you will have to have your land-line moved as well.

Mobile phones have no subscription right, and so are much easier to set up.  

Note: There is a type of cordless-phone/mobile-phone hybrid called PHS that used to be popular.  The advantage was that it was very cheap.  The disadvantage was that the reception was poor compared with normal mobiles, and it wouldn't work at high-speeds.  I won't discuss that here, but I believe Willcom still offers that service.

Major carriers offering normal mobile phone service include: NTT Docomo, Softbank, E-Mobile (Now Y!Mobile, a subsidiary of Softbank), and KDDI Mobile (aka Tuka), and Willcom (also a subsidiary of Softbank).  There are also MVNO carriers, such as OCN, Sony Mobile, and b-Mobile.  

Basically there are two ways you can get a normal mobile phone.  You can get a pre-paid unit, and then pay through the nose for minutes - or you can sign up for a normal contract.  If you are only in Japan temporarily, then you will have to go the pre-paid route, since all of the major carriers will only give contracts to Japanese residents.  

If you opt for the prepaid option, you will have to put down about $100-$200 USD for the phone, plus buy some minutes.  At least from Softbank, the minutes are available in denominations of roughly $30 and $50.  Some of the other carriers have stopped offering prepaid service all-together, claiming it is mainly used for criminal activities.  You may be able to buy a second-hand prepaid phone through Tokyo Message Board or Craigslist, and likewise when you are done with it, you can probably sell it there too.

If you go through the normal option, be prepared for some sticker shock.  The US companies usually lie to you about the phone price, claiming it's "free", or reducing the price by several hundred dollars.  Of course, the phones aren't really that cheap in the US, and consumers get ripped off by thinking they are... but we'll get to that in a minute.  An entry level phone will set you back about $400.  One of the fancier touch-screen phones with built-in TV (yes, they have that!) will be $800 or more.  You have the option to pay this all at once, or to have it split up into 24 monthly payments that get added to your bill.  Thus, a typical bill might look like this:

$40 Basic Service

$20 Phone Payment

$20 Usage charges (Minutes/Messages)

------

$80 Total

This is really how it works in the US too, since obviously they have to increase your monthly payment to pay for the phone itself.  The kicker is that in the US, since they don't call it a Phone Payment fee, it never goes away - that is, you never pay off your phone.  Indeed, in the US you actually do usually have the option of getting a new "free" phone when the service runs out, but you unfortunately don't have the option to have a lower bill.  (Europe works much the same way as Japan).  In Japan, once the phone is paid off after 2 years, that part of your bill disappears.

While many US and European carriers offer "Free Minutes", that generally doesn't apply in Japan.  Instead, there is a base rate you pay, and a per-minute or per-message rate in addition to that.  Typically you can pay a higher base-rate for a lower per-minute rate.

Another thing to note - Foreign phones in general don't work in Japan.  (iPhone 3G appears to be an exception to this).  Japanese phones in general don't work over-seas either.  Almost all Softbank models feature GSM compatibility, so they will work overseas, but most phones from the other carriers don't.

Also, speaking of the iPhone, it appears to be popular among foreigners, but I would suggest that's because they don't know what they are missing, with Japanese phones you can many things the iPhone doesn't have:

1. IR sending of contact information

2. DecoMail, etc.

3. Superior Japanese Input (and a keyboard, since Japanese typically use email much more than Westerners).

4. 1Seg mobile TV

5. Standard Japanese chargers (you can buy batteries at 7-11 and such if you need to juice up in an emergency)

6. Saifu Keitai (Phone Wallet)/Mobile Suica (allows you to use your phone to as a train pass and to pay various things)

7. Japanese phone applications (Many available only for the standard Japanese phone Java platform)

8. Proper Japanese Mobile Phone Web site support (no, safari doesn't work well for this)

9. QR Code Scanning

10. Some phones have 5, 8, or even 10 megapixel cameras and HD video recording...

11. TV Call - Yes, video calls on the cell phone!

and more...

One more thing to note.  Softbank is general the cheapest major carrier.  (MVNOs may be much cheaper, depending on the plan and your usage patterns).  NTT Docomo is more expensive, but generally has the best coverage if you live in the middle of nowhere.  Messages to people on the same carrier are usually free.

With cell phone calls costing upwards of $0.20 per minute or more, and more to the point, Japanese International Long distance rates being obscene, Voice-Over-IP is becoming an attractive option.  There are basically two routes you can follow:

a. Voice-Over-IP offered by your Internet Provider - In this plan, you sign up for VOIP with your ISP, and they will usually provide you with a box that you plug into the internet router, and then plug your normal phone into.  This is particularly great for grandmother types who don't want to figure out new-fangled technology, but might not mind having a reduced bill.  It's not the cheapest option, but the call quality should be high, since the ISP is running and supporting it after all.  (An example of this type of option is "Hikari Denwa" from NTT East or "dot 050" from OCN).

b. One of the various VoIP services offered by 3rd parties - My favorite is Skype, but there are plenty of others.  In this model, you use your software on your computer to make calls to normal phones for a small fee (or, to other computers for free).  With many of these services you can pay-as-you go instead of being locked into a monthly fee, which is great if you don't use it too often.  Also, you can pay to rent a number for Incoming calls, and it doesn't have to be a Japanese number.  For example, you can link a Japanese number and/or a US number to your Skype account for a relatively low monthly fee.  Then people in the US can call you for free.  Also note that if you are using the computer on both ends, then Google Talk (Now Hangouts) is another great option.

Note that Japanese numbers follow a pattern, in general, numbers beginning with:

03 are offices or homes in the Tokyo area

070, 080, and 090 are mobile phones or PHS phones.

050 are Voice-over-IP

(Other prefixes for other areas)

0120 indicates a free dial number (like 1-800 in the US).

Which means people will know when they call your Japanese number that it's voice-over-IP, but I don't see that as being a big problem.  What is a problem is that some Japanese phones are blocked against receiving calls from VoIP by default.  I take this as scare-mongering by threatened Japanese companies trying to tell customers that VoIP might be used by spammers.

Also, only 070, 080, and 090 numbers can send/receive SMS messages.  Since a lot of services (like LINE, etc,) require an SMS confirmation to sign up, it means you may have trouble signing up with a phone that can only do 050.

The great thing about Skype in particular is that there are many non-computer devices that support it, including the iPhone, Sony Mylo, Newer versions of the PSP, and other stand-alone wi-fi phones.  You can also buy a box that connects a traditional phone to your network as mentioned in scenario 1 above.  With the Mylo or Wi-Fi phones, they can be on stand-by 24 hours a day, which means they can receive incoming calls, making them great to actually replace a normal home phone.  (With, of course, the added capability of taking them wherever you go if there is an internet connection).  A bit limitation of Skype is that even if you pay for an "online number" for Japan, the outgoing caller ID is not reliable in Japan.

Besides Skype, Gizmo5 is another popular service.  Gizmo5 can be used with their computer client (similar to Skype), and also with SIP phones and boxes.  A SIP "ATA Box" is a special box that you plug into your internet connection, and then you can plug a normal phone into it.  It will act as a translator between the SIP protocol that Gizmo5 uses and your normal phone - meaning that your computer won't have to be on at all.  Of course, you can always use the PC client instead if you don't want to buy anything.  Gizmo5 is also similar to Skype in that you can have free PC to PC voice and video calls and/or paid incoming and outgoing calls to normal phones.  Gizmo lets you rent s US number, or but currently does not offer Japanese numbers.  Also note that if you use Google Voice, you can have a free US number, and connect it to Gizmo.  There are rumors of Google buying Gizmo, so this integration might improve in the future.  Note that Gizmo 5 has been shut down by Google. 

050 plus is a more phone-centric Skype alternative from NTT Communications, please check out more in our review at:

Yamanote Style 050 Plus Review

As mentioned above, Google Voice (Formerly Grand Central) is a phone service offered by google.  While it currently only handles US numbers, you can use it to send and receive SMS numbers with US cell phones from their web page, and it can receive voice mails which are emailed to you as a sound file and/or a transcription.  Since this number is offered free of charge for incoming calls, it may be a good choice for those who only need to use voicemail and/or SMS.  Although you can forward it to another phone number, it has to be a US number as well.  If you forward it to the above mentioned Gizmo5 SIP number, then you can receive unlimited incoming phone calls for free - but you have to be at your computer.   Note that Gizmo 5 has been shut down by Google, but incoming calls can be answered in the web browser with GMail if the proper plug-ins are installed.  Note that Google Voice can be used for SMS and voice calls from the computer (via GMail), and with some special applications for Android and iOS (As with Skype, the Android applications work for incoming voice calls, due to better due to integration limitations in iOS). 

You can use computer-only IM clients like MSN, Google Talk, iChat, AIM, and more, to do domestic and international voice calls for free - provided you want to be at the computer every time.  If you want to to be able to do normal voice calls from these clients, you can use GTalk2VOIP, which provides a paid, but cheap service for connecting these normal IP clients to the phone system.  Despite its name, GTalk2VOIP actually works with many different IM services.  Although their web page is less than fancy, it works, and the price is right.  They offer both US and Japanese numbers for a very cheap rate.  The Japanese numbers are also not area code 50 (VOIP) numbers, but rather Tokyo area code 3 (Office) numbers.  This means anyone in Japan can dial them, and they look familiar/legitimate.  An interesting combination is that you can combine GTalk2VOIP and Gizmo5, meaning that you can receive incoming calls on not only your computer, but your actual phone if you have a SIP ATA box.  Although this combination sounds complicated, it is cheaper than Skype for long term use.

Note while you can't typically easily use Fax services over voice-over-IP,  services like popfax.com offer outoing and even incoming faxes without the use of any fax machine.  You can upload the documents you want to fax as PDF files or similar, and then send them to any number you like for a small fee.  Alternatively, if you subscribe, they will set up an incoming number, and when a document is faxed to it, they will convert it to a PDF and email it to you.