Indeed. This is a pleasant exercise for the combined JRockit and HotSpot teams :-)

If you have followed the public announcement you already know that our plan is to carry both forward and merge the technologies over the long term. The exact method and timeframe is not clear yet. These are early days and we need to get the development teams time to do the due diligience - the code bases are quite different and both have strong and weak points. Luckily, our new ex-Sun colleagues have experience in merging code bases from previous acquisitions.

OpenJDK will remain the single open source Java and JVM implementation that Oracle contributes to. Open sourcing the current JRockit code base simply does not make sense. Better focus our efforts on the future unified offering which will be a merge of some kind between the two technologies.

Another thing worth mentioning here is that some features in JRockit do not make sense as part of an open source JVM. One example is integration with the Fusion Middleware logging infrastructure - definitely not of interest outside of an Oracle context. The open source codebase should not be polluted by such proprietary extensions. I don't have an answer to how to accomplish this yet, but it's possible that the modularity work in JDK7 can help.

I fully understand that the community is anxious to know the details of where this is heading but I must plead for patience. Our clear goal is to keep Java strong and vibrant for the benefit of Oracle, our customers, partners and the community (including our esteemed competitors). The role of steward of the Java community and provider of the reference implementation is new to us and we don't want to upset anyone by rocking the boat.

Regards,

Henrik Ståhl
Director PM, JRockit Products
speaking for the combined Oracle and Sun JVM teams

I found this thread in the ruby-lang mailing list very interesting. Next quotes are enlightening:

In particular, the GC algorithms in MRI and YARV are specifically designed with the assumption that they will never actually run in 99.999% of all cases. They are designed for scripting, where a script doesn't even allocate enough memory to trigger a collection, runs for a couple of seconds and then exits, after which the OS simply reclaims the memory: no GC needed.
That's why YARV and especially MRI are so exceptionally bad for server loads. It's also why REE can never be merged into mainline. (Jorg W. Mittag Aug 21 2010)

99.999% is a bit over-exaggerated, but it is true that garbage collection algorithm of YARV and MRI focus for throughput on non-memory extensive short-running programs, and GC of REE is not suitable for those programs. (Matz Aug 21 2010)
Scripting was the original use case for ruby, so that's understandable. It explains a lot.

The Java HotSpot VM also eliminates the concept of "handles," an indirection facility used to access objects in memory. This both reduces memory usage and speeds processing. "A traditional representation of objects is that when you have one object that points to another," says Stoutamire, "it has a pointer to the header of that other object. But the classic virtual machine uses a different representation, where, instead of pointing directly to the object, it points into a table. And the header of the object is in that table, as well as a pointer to where the fields of the object are."

Such indirection is particularly useful in terms of relocating objects in memory, which often comes into play during garbage collection (see below). "If object A conceptually points to object B," explains Stoutamire, "what it really does is point into the handle table. Suppose that object A points to the fourth entry in the table, and the fourth entry in the table has a pointer to object B. Now, when I move object B, all I have to do for object A is update the entry in the table. That's not so useful if I only have a single object pointing to B, but if I have 1000 objects pointing to it (or I'm not sure where all the pointers are), then all I have to do is update that single entry when I move object B."

While the use of handles brings greater ease of processing, such indirection is considerably slower, and the table takes up more space in memory. Further, the handle table's primary reason for being, to better facilitate object relocation in memory during garbage collection, has also since been shown to be less than necessary.

"In practice, it isn't much more expensive to simply update all pointers during object relocation," says Stoutamire. "In order to do proper garbage collection, you have to trace through the entire heap anyway—you have to go through every pointer. So the fact that you're visiting every pointer anyway, means that you have the opportunity to change it right then and there, without doing much extra work."

Finally, because the Java HotSpot VM uses direct memory references, it doesn't have to set up a memory-referencing handle when allocating memory, and it doesn't have to manage handles in addition to managing object memory. That makes the allocation of temporary data structures as fast as C's stack-based memory allocations, which is a big win.

......

Pre-Java HotSpot VM
Many Java virtual machines use "conservative" or partially-accurate collectors. A conservative collector assumes that anything that looks like a valid pointer may actually be a pointer. Conservative collectors are easier to implement, but a conservative collector doesn't always know for certain where all object references are in memory. As a result, it can, in rare instances, make mistakes—such as confusing an integer for an object pointer— which can create difficult-to-debug memory leaks.

Also, a conservative collector has to either use handles to refer indirectly to objects, or avoid relocating objects, since relocating handleless objects requires updating all references to the object and the conservative collector can't even be certain that an apparent reference is in fact real. This inability to relocate objects, in turn, results in memory fragmentation, and prevents the use of more sophisticated garbage collection algorithms.

Conservative garbage collection also has a negative impact in the realm of native methods. "A conservative garbage collector has to make sure that it doesn't move anything that's pointed to from outside of the Java code," explains Stoutamire. "And that means that you have to scan areas of memory, looking for pointers into your heap—and that all takes time." This was one of the problems faced in using Java's old Native Method Interface (NMI) specification. Ironically, this difficulty was solved through the use of a different kind of handles.

"With the Java 1.1 platform," says Stoutamire, NMI was replaced with Java Native Interface (JNI). With JNI, you never point from the outside to Java objects directly, you only point to handles, which then point to the objects. That means that the garbage collector doesn't have to worry about what's going on on the outside. But handles are only used to the extent that native code wants to point into the VM," he says. "It's no less efficient than the old way and allows for far more efficient garbage collection. So if you want to use the Java 2 platform and, hence, the Java HotSpot VM, you have to be using JNI."

By re-writing the Interpreter loop of the Java VM in assembly language, 1.1 now sports a VM that runs up to five times faster on certain operations, according to JavaSoft reports. Overall speed of the VM has also increased: up to two times the speed of the 1.0.2 VM on Solaris, and a little over 1.5 times as fast on Windows 95 and NT. Better performance, along with enhanced memory management, are the most important improvements in the 1.1 release, says Arthur van Hoff, former Senior Staff Engineer at JavaSoft, author of the original Java compiler and HotJava browser, now CTO for Marimba, Inc. "It is essential for our products that the VM is fast and does not grow indefinitely. The 1.1 release is significantly faster and implements a much better garbage collector, which even cleans up classes that are no longer referenced." With Sun's acquisition of LongView Technologies in February, the bionics may just be beginning for the Java VM. See the concluding section, "Planning for the Future," for more details.

Old (pre JDK 1.0.2) version of invokespecial was called invokenonvirtual
 Worked almost the same - but had a bug!
 One difference: invokespecial for a superclass with the super
flag set finds the nearest superclass version of method, not necessarily the specified one
 All files generated by a (post-1.0.2) Java compiler should set super flag

As promised, here's my review of the new JRockit book...


Having used JRockit for years (mostly sitting beneath WebLogic) I've been waiting for something like this book to arrive and lift the lid off the JVM to show what's inside. I'm glad to say that I am not disappointed. This book is a must-have for any serious enterprise Java developer or administrator who uses JRockit and wants to better understand how to tune, manage and monitor the JVM. The book also provides the Java developer with a clearer direction on the do's and don't's for developing a well behaving application on top of JRockit, especially with regards to memory management and thread management. Even for users of other JVMs, like Hotspot, much of the first half of the book, which concentrates on the JVM's internals, is relevant and insightful into the workings of Java virtual machines generally.

The first half of the book concentrates on providing in-depth knowledge of JRockit's internals and focusses on:

    * Code Generation (Java bytecode, generation of native code, JRockit 's Just In Time compilation strategy with subsequent optimised code re-generation)
    * Memory Management (heap usage, thread-local allocation, garbage collection strategies, deterministic-GC, 32-bit vs 64-bit memory management, potential pitfalls)
    * Threads and Synchronisation (green vs O.S. threads, synchronisation patterns/anti-patterns, code generation strategies for locking/unlocking, thin and fat locks, lock profiling)
    * Benchmarking and Tuning (throughput vs latency, tuning tips, benchmarking tips)

Don't be put off by the early exposure to Java bytecode samples in the book. The bytecode examples are only used liberally and only in a couple of early chapters. It's worth reading these bytecode samples carefully because the points these chapters make will resonate more strongly if you do.

As I read this book, it became evident that the JRockit engineers are extremely passionate about their work and that they live and breath it. It is re-assuring to know that they are constantly striving to improve the product, based on deep scientific theory yet always with an eye on pragmatism and real-world usage. I'm sure this is why JRockit is fast and powerful whilst reminding easy to manage and run applications on. Throughout the book, the ethos of JRockit's design is very clear. It's an adaptive JVM, where the internal runtime is constantly and automatically being re-evaluated and re-tuned, according to the current nature of the hosted application's usage and load. Reading the book, I can better appreciate the argument that server-side Java is faster than an equivalent native C application. A native application only has one-shot, at compile time, to generate optimal native code. JRockit, however, takes the opportunity to revisit this at runtime, when it has a much better idea of actual usage patterns.

The second half of the book provides an introduction and detailed reference into JRockit's rich and powerful management and monitoring toolset, including:

    * JRockit Mission Control (JRMC), composed of the Management Console, the Flight Recorder (plus Runtime Analyzer which it replaces) and the Memory Leak Detector
    * JRockit Command line tool (JRCMD), including in-depth examples of all the important commands
    * JRockit Management APIs, including the Java API for direct in-line access to the JRockit JVM from hosted Java applications plus the JMX version of the API for remote access to the JRockit JVM

These chapters provide an easy to read introduction to the key features of JRockit's rich tools, lowering the barrier to use for newbies. Also, many of the illustrated examples are use-case driven, acting as a handy reference guide to come back to at a later date. For example, if you suspect that you have a memory leak in your application and want to work out how best to locate the root of the leak, using the tools, the Leak Detector chapter will take you by the hand through this process.

Finally, at the end of the book there is an introductory chapter covering JRockit Virtual Edition (VE). JRockit VE enables the JRockit JVM, accompanied by a small JRockit kernel, to be run in a virtualised environment. This runs directly on-top of a hypervisor (eg. Oracle VM) without requiring the use of a conventional, heavy-weight, general-purpose operations system such as Linux, Solaris or Windows. Such a fully functional operating systems would otherwise burden the system with a layer of unwanted latency. This chapter in particular makes me realise that the JRockit Engineers are metaphoric Rocket Scientists (or should that be Rockit Scientists? ). I defy anyone not to be impressed by the ambition of JRockit VE and and the high level of technical expertise that must have gone into developing it!

To summarise, I highly recommend this book. Easy to read yet very insightful. Once you've read it, it will remain as a handy reference to come back to, especially when needing to use the JRMC tools to diagnose issues and tune the JVM and hosted applications.

好贵的书啊  $71.39

The obvious solution of placing the header alongside the mark bit would incur false sharing when the GC thread writes to the mark bit and a mutator reads the header information.

This free book is a collection of notes and sample codes written by the author while he was learning JVM himself. Topics include JVM (Java Virtual Machine), HotSpot, JRockit, GC (Garbage Collection), Memory, Stack overflow, CDS (Class Data Sharing), Runtime, Reflection.

Java is a very hard language to bootstrap. It is even harder to bootstrap with "official" JDK and set of required VM features. Most of this is due to the lazy initialization and dynamic class loading model but a large part is also the poorly factored design of the JDK classes.

There is a fundamental tension between object-oriented design
and lock-based concurrency control
• OO encourages you to hide implementation details
• Good OO design encourages composition
But...composing thread-safe objects requiring knowing
implementation details
• So you can participate in other object's locking protocols
• So you can avoid deadlock
• Language hides these details, leaving users to rely on hidden information
• If that information changes out from under you, you're sunk

> How does this compare with other "fast" garbage
> collector's such as the Jamaica RTSJ JVM? (not quite a
> straight comparison as that's RTSJ rather than straight
> Java but still...)

They live in different application spaces. Jamaica, like several other RTSJ implementations, focuses on hard real time realtime systems (typically in the embedded world). See http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.62.5942&rep=rep1&type=pdf. I would not characterize it as a "fast" GC - instead I would characterize is as an extremely consistent GC.
Jamaica identifies the universally recognized problem of heap fragmentation (there is no way to stop fragmentation from happening in contiguous Java heaps). In order to combat fragmentation without using compaction, such systems often use new heap and object layouts. In Jamaica's case, the heap is broken into fixed sized blocks [32 bytes in Jamaica's case] and Java objects larger than 32 byte end up chaining discontiguous blocks together (in either linked lists for objects, or trees for arrays). This achieves the hard real time requirement by guaranteeing no compaction will ever be needed, but it comes at a very steep memory access costs (in both memory op counts and cache misses, and in the loss of optimization opportunities and the ability to directly use common CPU addressing modes). It is exactly the right tradeoff for hard real time. Real time is not about being fast, it's about being slow but predictable.

java version "1.6.0_20"
Java(TM) SE Runtime Environment (build 1.6.0_20-b02)
Java HotSpot(TM) 64-Bit Server VM (build 16.3-b01, mixed mode)

ubuntu 10.04 LTS 64-bit

You must invoke jstack as the user who started/is running the given process.
The above error occurs if you invoke jstack as "root"/via sudo (Has Super Cow Powers but is still the wrong user)

I guess you tried the "-F" option, because it didn't work without it. Therefore, you tried the sledgehammer. "-F" IS for a hung process.

If you want to create a thread dump of e.g. tomcat6 (installed from ubuntu package repo) you must type the following:

"sudo -u tomcat6 jstack 8812"

Executes jstack as user tomcat6, who is running the service tomcat6, assuming tomcat's pid is 8812.

This is not an ubuntu bug, it's - say - some sort of usability issue with this jstack thing. Blame Sun/Oracle . . .

The Java version information can confirm that Compressed
references are in use. The "GC" Line of the version output will
display "CMPRSS" after the build date, similar to:
GC   - 20100308_AA_CMPRSS)

The solution to this performance issue is to be smarter about the data structures. This is exactly what we did in the IBM Java6 JDK with the compressed references feature. We can play tricks (and not get caught) because the user (java programmer) doesn’t know the internal representation of the java objects.

Compressed references are enabled through the “–Xcompressedrefs”
command line switch in the IBM J9 for Java 6 JVM. This option is set
automatically in supported 64-bit releases of WAS V7, depending on
whether the maximum Java heap size is less than 25GB. While CR can
address up to 28GB, the 25GB value was chosen as a safe parameter for
automatic enablement across disparate platforms. Details surrounding
this value have been provided in the CR technical details. To explicitly
disable CR, WAS can be started using the “-Xnocompressedrefs”
option.
WAS V7 is supported on the majority of commercial 64-bit platforms (i.e.
POWER, x86-64 from Intel/AMD, Sun SPARC, etc), however in order to
use CR technology the WAS release must be packaged with the IBM J9
for Java 6 JVM. Please refer to the WAS V7 information center (4) for
detailed and up-to-date software and hardware system requirements.


When running 64-bit IBM J9 for Java 6 in CR mode, the JIT compiler
detects the specified maximum heap size and applies appropriate
compression and decompression. Namely, if the maximum heap address
used by the Java JVM process is below 4GB, the shift operation on
compression and decompression is avoided. Figure 2 provides a
graphical representation of the address reference models used in each
addressing mode.

Initial Implementation
No restriction on placement of Java heap in address space (Heap_base)
Compression is subtract: (64-bit address – Heap_base)
Decompression is add: (32-bit offset + Heap_base)

Significant overhead on some platforms
Increase in path length due to add/sub
Need to handle NULL values

===================================

Implementation in IBM JDK for Java6

Java heap placed in 0-4GB range of address space
32-bit offset in object is simply the address in 0-4GB range

Compression (64-bit   →   32-bit) : nop
Decompression (32-bit   →   64-bit) : zero-extension

Maximum allowable heap in theory is 4GB
In practice, the maximum heap available is lower
~2GB on Windows and zOS

Option to enable compression in 64-bit Java JDK
-Xcompressedrefs

----------------

Java objects in J9 are 8byte aligned (low 3 bits are 0)
Main idea is to store 32-bit shifted offset in objects
Address range restrictions relaxed
Java heap allocated in 0-32GB range

Compression (64-bit   →   32-bit) : right shift
Decompression (32-bit   →   64-bit) : left shift

Maximum allowable heap in theory is 32GB
In practice, the maximum heap available is ~28GB

The sampler thread turns the profiler OFF or back ON
Number of consecutive ticks in JIT generated code turns the profiler OFF
Number of consecutive ticks in interpreter turns the profiler back ON

This paper describes optimization techniques recently applied to the Just-In-Time compilers that are part of the IBM® Developer Kit for JavaTM and the J9 Java virtual machine specification. It focusses primarily on those optimizations that improved server and middleware performance. Large server and middleware applications written in the Java programming language present a variety of performance challenges to virtual machines (VMs) and justin-time (JIT) compilers; we must address not only steady-state performance but also start-up time. In this paper, we describe 12 optimizations that have been implemented in IBM products because they improve the performance and scalability of these types of applications. These optimizations reduce, for example, the overhead of synchronization, object allocation, and some commonly used Java class library calls. We also describe techniques to address server start-up time, such as recompilation strategies. The experimental results show that the optimizations we discuss in this paper improve the performance of applications such as SPECjbb2000 and SPECjAppServer2002 by as much as 10-15%.

64-bit SDK for z/OS, Java Technology Edition Version 6 Release 0 Modification 1 is a Java SDK that contains a reengineered Java virtual machine at the SDK 6 level and the IBM Just-In-Time (JIT) compiler. The program is a key building block for developing on-demand applications. 64-bit SDK for z/OS, Java Technology Edition Version 6 Release 0 Modification 1 is compliant with the Java SDK 6 compatibility test and is designed to provide the stability, service, scalability, and integration you expect from a System z program.
64-bit SDK for z/OS, Java Technology Edition Version 6 Release 0 Modification 1 is designed to provide:
Support for Java application programming in a 64-bit application environment
Java APIs at the SDK 6 level
Continuation of the "write once, run anywhere" Java paradigm
JZOS support and enhancements
Exploitation of z196 instructions
Added z/OS Java unique security functions
XML support
System Authorization Facility (SAF) and cryptography support
Use of your System z Application Assist Processors (zAAPs) available on z196, z10 EC, z10 BC, z9 EC, z9 BC, z990, and z890 servers to run eligible Java work
Additional reliability, availability, and serviceability (RAS) enhancements
z/OS Java unique security
The following z/OS security and cryptography enhancements have been added to 64-bit SDK for z/OS, Java Technology Edition Version 6 Release 0 Modification 1:
Support is added for AES Secure Keys in the IBMJCECCA hardware crypto provider. This capability requires z/OS V1.11 and ICSF HCR7751.
Enhanced ICSF exception handling is added in the IBMJCECCA hardware crypto provider, helping to simplify and streamline problem determination.
JZOS functions
The following JZOS enhancements are added to 64-bit SDK for z/OS, Java Technology Edition Version 6 Release 0 Modification 1 to help to:
Enable the submission of jobs to the MVS™ internal reader and retrieval of the submitted JOB ID. Text, record, and binary JCL will be supported. The MVS internal reader attributes LRECL, RECFM, and Class will be configurable from this Java class.
Enhance the Zlogstream class to support IXGBRWSE (read) and IXGDELT (delete) and add InputStream/OutputStream Java wrappers.
Support Extended Address Volumes (EAV).
Add a new package -- com.ibm.jzos.wlm -- with selected z/OS Workload Manager (WLM) APIs.
These JZOS functions have been previously available from the IBM alphaWorks® website, and are now integrated into the IBM z/OS Java SDK. For reference, the IBM alphaWorks website is located at
http://www.alphaworks.ibm.com/tech/zosjavabatchtk
RAS
64-bit SDK for z/OS, Java Technology Edition Version 6 Release 0 Modification 1 includes similar dump capability, tracing, and problem determination as in SDK for z/OS, Java Technology Edition Version 6 Release 0 Modification 0.
Section 508 of the US Rehabilitation Act
64-bit SDK for z/OS, Java Technology Edition Version 6 Release 0 Modification 1 is capable as of March 15, 2011, when used in accordance with IBM's associated documentation, of satisfying the applicable requirements of Section 508 of the Rehabilitation Act, provided that any assistive technology used with the product properly interoperates with it. A US Section 508 Voluntary Product Accessibility Template (VPAT) can be requested on the following website
http://www-03.ibm.com/able/product_accessibility/index.html

14. Formed OTI, an embedded Smalltalk company Digitalk 32-
bit Smalltalk, Embedded Smalltalk for Tektronix, Just in
Time Software, ENVY/Developer, Actra, HP NA, Momenta
Pen computer, Sony Qualcomm phone, Siemens PBX, IBM
Smalltalk and VisualAge K8 VM
15. Enter Java...sold OTI to IBM, built VisualAge Java (UVM)
and Eclipse, IBM Pervasive Computing J9 VM