This is really pretty simple.
If you're targeting desktops, you should use Java. It's the only environment well ported to all of the major desktops, and it works surprisingly well to the 90-95 percentile. You can create perfectly usable GUIs and have access to a rich graphics rendering framework. It also can drive printers reasonably well, and portably.
As long as you're not trying to create the latest McGreatest action 3D shooter, it's more than adequate. The complaint about downloading the JRE is a red herring, as it's pretty much painless to install on all of the platforms, including Linux. And once the JRE is installed, publishing for the major platform is mostly automated at this point (using simple packaging utilities), and with WebStart, it's even more simple, and cross platform.
Java lacks specifically in being a "perfect citizen" with all of the desktop environments. Out of the box, generic Swing works well enough though that the benefits far outweigh the downsides.
If you're targeting a hosted solution, you can use whatever you're happy with and target the modern, rich Web experience. Modern browsers are still bleeding edge, even with the cross browser libraries. Arguably, the cross browser stuff is more of a pain to manage and debug than Java is to work with. But if you want to target the browser, that's simply the truth of it.
You could almost safely target a single browser, Firefox, since it runs everywhere and is certainly a leading browser. That can save the pain of worrying about cross browser details. Some will decry its lame to target a single browser, but they're not the ones writing and hosting software for a niche environment.
However, if you want to target mobile, then Firefox is not necessarily the best development browser, as most of the mobile browsers are based on WebKit (save for Windows, of course...). In that case you're better developing in Safari or Chrome.
Also, all of the new browsers do support the concept of the "off-line" application. Where the bulk of the resources are cached locally on the mobile device or desktop computer. You can have local databases, etc. with these applications, and they don't need network connectivity to run. This area is a bit bleeding edge right now, but it's possible.
If you're writing a hosted app, then the backed environment that is most popular is simply PHP. The major benefit is that it's dirt cheap to host, and performs reasonably well on cheap hosting servers. Most hosts have real good support for PHP above pretty much anything else. You can still use CGI and other scripting languages (Perl, Python, etc.), and some host will support them well (with mod_perl, etc.), but PHP is by far the most dominant in this domain.
Another alternative is the Google App Engine. Here you can develop in Java or Python, and the hosting is effectively free, but it's definitely a different development model that most folks are unaccustomed too, so it takes some getting used to. But it offers some nice advantages.
If you want to target mobile directly, then you have to be specific and simply pick one. The modern mobile space is similar to the old school world of Commodores, Apples, Ataris and nascent IBM PCs with old school CP/M machines. This world is rocketing forward and has yet to stabilize. Obviously, Apples iOS and Android are the heavy hitters here, with Android likely grabbing more overall market share, but, at the same time, with a wide mix of platforms and devices as well. Android is certainly easier for the casual developer since you can publish your own code (i.e. you don't have to use the official app store for Android) and you can develop on anything you want (PC, Mac, Linux).
