Ordering of getFields(), getMethods(), getDeclaredFields(), etc. - Java SE (Archived)

The documentation says that the values returned by getFields() etc. are not returned in any particular order.
It would really be helpful, for certain applications of reflection based code, if they would be guaranteed to be returned in the order they were declared in the source file.

But the runtime only knows the order that they are in, in the bytecode, and jikes seems to have a different order in the bytecode than in the sourcecode (javac seems to keep the order intact)
You could maybe work it out from the linenumber table IF THAT WAS PRESENT in the bytecode, but it isn't always.
What if the class was compiled from some other language where there was more than one file (compilation unit), contributing to the class (AOP aspect injection at class loading time comes to mind here), what should the order be?

Right.  That's why this is a feature I'd like to see in a future version of java.
With that feature you could, using only reflection, take a simple class and write it out to a socket as if it were a C struct, figuring out byte padding, etc.
You could similarly serialize a class to a CSV file using this mechanism (if you could rely on the order).
Perhaps rather than making them all return the declared order, one could have a tag #OrderedFields or something that would mark a class as having this property.

If this is a requirement, I think your suggestion of using a custom annotation to specify an ordering is the correct way to go.
If you really must have source code ordering (rather than a deterministic order), an "apt" annotation processor could use the source line position to determine the order.
Also, if you are using JNI to access C Structs, maybe a better option is to generate a java wrapper from the C source. I have done this (almost), I generate java source from a simple text file that basically contains the C-Struct information (in this case it was easier to re-write the information, than to find/implement a c parser).

I was thinking of the annotation being a special one the compiler understood... hacking this up with apt would be awkward.  I'd have to either tag every field, or parse the source code once I saw the tag on the class.
In my example I was not thinking of JNI, I was thinking about having java talk to a C program over a socket where the C program just writes a struct out.
But how hard would it really be for the compiler writers to just keep the source order in the class file?  Naively, I think it would be simple.

OK, I was ignorant!
If you compile with apt (instead of javac), you can get all the fields in the source and sort them by line number.
I didn't realize how powerful apt is!

One more convert  :), one less Mustang RFE, so SUN can get on with the stuff we can't do ourselves :)

Related

How to create a scripting language in java?

Hello,
All that I want to do is to create a scripting language in java. I�m familiar with javacc, jsr223 and other things but don�t know how to start. The language syntax is java 5 syntax with some change and I want to generate java source code from small scripts. In fact I don�t know how other languages (like groovy) are created.
I hope you can show me the required steps.
Looking forward to hear from you.
Thanks. 
That's all, huh?
For a start, generally when I hear "scripting language" I'm thinking interpretter, not a system which creates compiled (or compilable) code.
I get the impression that what you're talking about is what I'd call a "preprocessor" language, some extra syntax added to java which the preprocessor renders into ordinary java.
As far as complexity is concerned, much depends on how deeply involved the extra syntax is with the embedded ordinary Java. Does you preprocessor need to understand the java, or is it just embedded as text? How easilly identified are your preprocesor statments? It's a lot easier to do this if preprocessor lines are instantly identifiable from Java code (e.g. start with a #).
Basically the stages are always the same;
1) Lexical analysis i.e. picking out words, operators, numbers, quoted strings.
2) Construct a syntax tree.
3) Generate code (in this case Java).
Now, in this case, some of the nodes in the syntax tree may simply be chunks of undigested Java. 
From the sound of it, this library is your missing link:
http://cglib.sourceforge.net/ 
Thanks for replies.
Let me explain a little more.
Yes it is interpreter. The scripting language will let to code just like java but can use some new features and there is no limitation for using java codes. New features are not with some special characters. For example there is an easy way for adding an event listener and also there are two new keywords.
It seems I have to use a parser, is it right? Then which compiler compiler do you suggest. Yacc,antlr or javacc?
Regarding http://cglib.sourceforge.net/ it seems that this is for manipulating class files. I want to extend java features not classes. 
Regarding http://cglib.sourceforge.net/ it seems that
this is for manipulating class files. I want to
extend java features not classes.And how do you image one achieves that? Take a look at BCEL also (implemented using CGLIB). 
No, if it generates class or java files it's a compiler or preprocessor. An interpretter is when, having parsed the source, it's executed from the syntax tree without being translated to an executable form.
The natural lexical analyser to use would probably be InputStreamTokenizer. The details of syntax analysis is heavilly dependant on the syntax you're proposing. 
I would debate your analysis here - "without being translated to an executable form."
A class file is not directly executable. This goes double for a class "file" that never exists in the file system, but rather is generated to, and loaded from, heap memory.
If you take the narrow view that a class can never exist as "executable code" in any sense, then any interpreter with a JIT stage fails your definition. 
To the OP, however, what you're describing does sound more like what I would think of as a "compiler", in that its primary output is a .class or .java file.
Either way the CGLIB/BCEL libraries will be useful in generating your output. JavaCC or ANTLR are probably your best bets for parsing your input.
Dave. 
It's supposed to give us java source file and a bunch of other files. Then user can compile and use the java files. It will be possible to use other java classes (and every features of java) from within the language. Maybe it's something like a pre compiler.
meybe a btter question is how groovy is created? I know they have antlr grammer file but is it enough?
Thanks. 
Yiu could have a look at Groovy, a scripting language based on the JVM. 
Yiu could have a look at Groovy, a scripting language
based on the JVM.I see now you already know about it. 
I would debate your analysis here - "without being
translated to an executable form."
A class file is not directly executable. This goes
double for a class "file" that never exists in the
file system, but rather is generated to, and loaded
from, heap memory.Well, strictly speaking javac is a "Semicompiler", but neverthless class files are the "machine language" for a machine which is usually emulated by the JVM. It's possible in principal (and I believe Sun put some work into this) to produce a machine for which the Class file code is implemented in hardware.
Typically an interpretter, though, works at the syntax tree level. Variables are still looked up by name where used. (Which is why you can generated variable names dynamically).

Compiling new classes on the fly without having a .java file

Hi everybody,
I'd like to compile dynamically produced JAVA code (code from user input in my application). I don't wanna store the source code in a .java file, but just compile the code (as char[] or something like that) to a byte[].
Does anybody have an idea ??
Thanks
Jan
Are you planning to write a Java compiler?
If yes, I think you should have a JIT implementation first, but I am not sure. Please refer to java.lang.Compiler.
Otherwise, you could use javac.exe as your tool. Start it by System.exec(), let it working on buffer files.
This is not the answer, it is simply my thoughts before. If you have answer, pls tell me.
Sure, you can generate your own byte code. And you can put it into a byte array and load it as a class. What idea do you want? 
yes, i've heard of that ... but how do I do it ?? I want that user to write little code fragments (I will add the method an class stuff) and then the code should run on the VM ...
do you have any idea ??
Thanks ...
Look at the sun.tools.javac.Main (and other classes in the sun.tools.javac package). You'll need tools.jar in your classpath. There is very little documentation about these non-standard classes, but they should do what you want. 
Look at the sun.tools.javac.Main (and other classes in the sun.tools.javac package). You'll need tools.jar in your classpath. There is very little documentation about these non-standard classes, but they should do what you want. 
Have a peek at BCEL http://jakarta.apache.org/bcel/index.html
It provides sufficient functionality to manipulate and create class files at runtime.
Good luck.
Peter 
I really don't think you want to do that.
What is the problem you are trying to solve?
How will you debug the code (which doesn't exist except on a transient basis)?
How will you assure that you generate a valid class or set of bytecodes every time?
Dynamically generated code generally ends up being the most painful solution imaginable, because supporting it is very difficult. 
However, I really wanna do that. The user should write very tiny code fragments like :
if (a > 20) return 42;
else return 43;
I will wrap the code in a method and class and compile it. I load this class with a custom classloder. If it is avoidable, I don't wanna work with any files... 
Have you looked at the idea of interpreting rather than compiling, say using Jython (a python interpreter written in Java). Python and Java are extremely close in syntax, and it should be relatively easy to do what you are talking about.
I am worried though, that you are going to let users type in text, and assume that it has valid syntax in the language you are using. Many years of UI experience indicate that letting users type in anything results in a certain error rate. Letting it be instructions you are to follow in your code is even more dangerous.
There are many programs that let users define actions interactively. They seem to either let the user type in text, and error check it to death (a lot of work, think spreadsheet type programs), or they present the user with a set of choices, often using a wizard type interface, which takes more up front UI work, but only lets the users select valid options.
It sounds like you want to interpret what the user types in as instructions. Even with Jython (or a similar interpreter), you would need to parse the user input for formatting errors, invalid variable names, and anything else they could make a mistake on, before you could pass any of it to the interpreter.
Good luck. 
However, I really wanna do that. The user should write
very tiny code fragments like :
if (a > 20) return 42;
else return 43;
I will wrap the code in a method and class and compile
it. I load this class with a custom classloder. If it
is avoidable, I don't wanna work with any files... IBM's VisualAge for Java does this via its Clipboard feature.
I think your best bet is to generate the the code in a StringBuffer and then save it out to a temporary file. Then call sun.tools.javac.Main to compile it to a .class file. Now you are ready to load it with its own ClassLoader instance (try URLClassloader using the file URL as the classpath to find your temporary class file) and execute it. When you are done delete the files and unreference the ClassLoader and temporary class and all should get cleaned up.
One problem that I have read about somewhere is that sun.tools.javac.Main may leak memory if you are not careful (this should be fixed in future versions of the javac). 
However, I really wanna do that. The user should write
very tiny code fragments like :
if (a > 20) return 42;
else return 43;
I will wrap the code in a method and class and compile
it. I load this class with a custom classloder. If it
is avoidable, I don't wanna work with any files... The perfect tool for this is BeanShell (http://www.beanshell.org).
You just create a BeanShell interpreter object (new bsh.Interpreter()), and call Interpreter.eval() on the expression (after binding any variables like "a" to some values on entry).
why not go all the way? skip compiling and code straight into a classLoader? 
Look at this:
http://www.esus.com/docs/GetQuestionPage.jsp?uid=410
Look at this:
http://www.esus.com/docs/GetQuestionPage.jsp?uid=410

Printing annotations; readability of package annotations

Hey,
A couple of JSR 269 things. First, is there a way to get correct source code for an annotation object returned by Element.getAnnotation for printing to the writer of a source file, or do you have to get the AnnotationMirror for that? When appropriate I tend to use plain annotation objects instead of mirrors since they're more convenient to work with, and a convenient way to get compilable code for them would be nice. As an aside, Elements.getConstantExpression doesn't seem to throw a IllegalArgumentException when "the argument is not a primitive value or string", but rather the toString of it, which doesn't give compilable code for at least annotation objects (hey I had to try, despite what the doc says)
Next, related to the reason I'm tinkering about with printing annotations in the first place. As far as I can see invoking getAnnotation and getAnnotationMirrors on package elements always return nothing if the corresponding package-info.java source files are not on the list of source files to be compiled being given to javac. This is different from annotations on types, which may be readable even when they're not being compiled (depending on Xprefer and the retention policy of the annotations, of course) if you somehow get their element. Is this the correct behavior? I'm using 6u10.
Thanks in advance. 
thormick wrote:
Hey,
A couple of JSR 269 things. First, is there a way to get correct source code for an annotation object returned by Element.getAnnotation for printing to the writer of a source file, or do you have to get the AnnotationMirror for that? When appropriate I tend to use plain annotation objects instead of mirrors since they're more convenient to work with, and a convenient way to get compilable code for them would be nice. As an aside, Elements.getConstantExpression doesn't seem to throw a IllegalArgumentException when "the argument is not a primitive value or string", but rather the toString of it, which doesn't give compilable code for at least annotation objects (hey I had to try, despite what the doc says)You might like to look at project Hickory 's [Prism |https://hickory.dev.java.net/nonav/apidocs/index.html?net/java/dev/hickory/prism/package-summary.html] mechanism as sweet spot between Annotations returned from getAnnotation, and AnnotationMirror.
In regards to extracting the source code, I wouldn't expect an Annotation returned by Element.getAnnotation to necessarily have a toString() method that returned the source code. And there is no other defined method that would. Many of the Element implementations do return the effective source code when their toString() is called, though again that is not defined.
If the AnnotationMirror.toString() gives you what you want, but you would like the convenience of an Annotation returned by Element.getAnnotation, then have a try of the Prism mechanism, generate a Prism class for your know annotation, and from that you can get the values in a convenient form (more convenient than a Annotation), and still access the underlying AnnotationMirror.
Bruce 
thormick wrote:
Hey,
A couple of JSR 269 things. First, is there a way to get correct source code for an annotation object returned by Element.getAnnotation for printing to the writer of a source file, or do you have to get the AnnotationMirror for that? The printing utility included as part of JSR 269, javax.lang.model.util.Elements.printElements acts on Elements, not annotations; although it will print annotation information.
Without taking imports into account, correct source code will need to use fully qualified names, which may not be what you want.
When appropriate I tend to use plain annotation objects instead of mirrors since they're more convenient to work with, and a convenient way to get compilable code for them would be nice. As an aside, Elements.getConstantExpression doesn't seem to throw a IllegalArgumentException when "the argument is not a primitive value or string", but rather the toString of it, which doesn't give compilable code for at least annotation objects (hey I had to try, despite what the doc says)Yep, that is bug
6517779 javax.lang.model.util.Elements.getConstantExpression() doesn't throw any exception
>
Next, related to the reason I'm tinkering about with printing annotations in the first place. As far as I can see invoking getAnnotation and getAnnotationMirrors on package elements always return nothing if the corresponding package-info.java source files are not on the list of source files to be compiled being given to javac. This is different from annotations on types, which may be readable even when they're not being compiled (depending on Xprefer and the retention policy of the annotations, of course) if you somehow get their element. Is this the correct behavior? I'm using 6u10.It is certainly possible that the implementation's handling of a package-info.class may be a bit buggy compared to handling the source version. Note that unless the package annotations have at least class file retention, there might not be any package-info.class file. 
brucechapman wrote:
-snip-
If the AnnotationMirror.toString() gives you what you want, but you would like the convenience of an Annotation returned by Element.getAnnotation, then have a try of the Prism mechanism, generate a Prism class for your know annotation, and from that you can get the values in a convenient form (more convenient than a Annotation), and still access the underlying AnnotationMirror.
BruceOh thanks! I was aware of Hickory but haven't looked at it properly.
j.d.darcy wrote:
It is certainly possible that the implementation's handling of a package-info.class may be a bit buggy compared to handling the source version. Note that unless the package annotations have at least class file retention, there might not be any package-info.class file.That was my first thought too, that it's something with how package-info.class is handled, but I've also noticed the same result on a fresh, partial compile. I cleaned out the class destination directory and with "sourcepath" set I compiled only a single class so that my annotation processor would get triggered, but it still couldn't see any annotations I had placed on the package while it could see those on other classes. Yeah, I know about retention, this happens no matter what it's set to.
Well thanks either way. I think I'll rework my processor so it doesn't rely on package annotations being readable when compiling classes. 
I have run into the same issue with my annotation processor not being able to see package annotations that are declared in the package-info.java file. The Elcipse IDE says that package annotations must go in package-info.java but I don't see how I can make use of these annotations. Is this info treated as implicitly compiled and not subject to annotaiton processing? If you specify the package-info.java file on the command-line then the processor can see these files, but this is not a viable option.

Bytecode modification

Hi there,
We are trying to inject errors at bytecode level. Is it possible to get some useful information like some sort of pattern from the java class file without using any tool like BCEL or similar + Is it possible to do some hex (cafebabe0000002e001b0a0005000e09000f00100) encoding and decoding (i.e. reading and writing) on java class file without violating the class file format? Any comments, suggestions.... 
Sounds to me like you're too lazy to learn BCEL or similar....
There's no way to "inject errors" into a class method's bytecode without explicit knowledge of bytecode, instruction formats etc.
Well, it's not a matter of laziness :). Learning Java API's has never been or is a problem for me. The only thing i want to know is that whether we can manilpulate bytecode which is input data for the JVM i.e. the Hex info in Java class file.
""There's no way to "inject errors" into a class method's bytecode without explicit knowledge of bytecode, instruction formats etc.""
Do you have any evidence or proof for "no way". 
i think i should have asked this question on the first place.....
is it possible to manipulate bytecode without disassebling ( here comes BCEL :) ) it? If yes, how... If not why.....
Thanks. 
We are trying to inject errors at bytecode level. Is
it possible to get some useful information like some
sort of pattern from the java class file without using
any tool like BCEL or similar + Is it possible to do
some hex (cafebabe0000002e001b0a0005000e09000f00100)
encoding and decoding (i.e. reading and writing) on
java class file without violating the class file
format? Any comments, suggestions....I once wrote a small utility (in C) to parse bytecode to human-readable
because the output from javap was 'not good enough'.
The class file format is not all that difficult.
You could easily parse and rewrite a class file
injecting or changing (semi)random bytes
without breaking the class file format. 
is it possible to manipulate bytecode without
disassebling ( here comes BCEL :) ) it? If yes, how...
If not why.....If you just want to load random bytecode as a class definition (maybe starting with a known class definition and corrupting it somehow), you can always create a custom classload and override defineClass() to load it from your buffer.
Basically, I have no idea what you mean when you say "manipulate". If you describe in a little more detail exactly what you're manipulating, and how you would like to do it, we can help you better.
Well, it's not a matter of laziness :). Learning Java
API's has never been or is a problem for me. The only
thing i want to know is that whether we can
manilpulate bytecode which is input data for the JVM
i.e. the Hex info in Java class file.You can of course create your own version of BCEL.
Of course you could write your own VM as well and that would be another solution.
Seems pointless to me given that BCEL exists.
""There's no way to "inject errors" into a class
method's bytecode without explicit knowledge of
bytecode, instruction formats etc.""
Do you have any evidence or proof for "no way".Of course you can.
You could simply get the class file length and then zero fill the entire file. Or zero fill the first x bytes.
Of course that would produce exactly the same verification error every single time, but if that is all you want then go for it.
On the other hand if you want to insert different type of legal errors into the code then you are going to have to learn about byte codes and the java class file format, regardless of how you implement the solution.
Thanks every1 who has contributed with his/her thaughts, ideas. The issue has been resolved.

[b]Reflection Question - Complex (Answer if u can ?)[/b]

In C# within the reflection.emit package there is a way to generate a class definition on the fly at runtime, emit the byte codes that constitute the compiled class and able to invoke methods on that class.
I am looking for similar functionality to implement in Java. Using a custom class loader I can create a class instance of let us say foo if the bytecodes are already there but I am not able to figure out how to create the bytecodes (for foo) on the fly. I hope the question is clear. I know this has security conotations and the JVM might not even allow this? 
Byte codes are just bytes, you use ordinary IO tools like FileOutputStream to write them to a file or ByteArrayOutputStream to write them to an array of bytes. But is your question, how do you know which bytes to write? Well, that's the job of a compiler. If you want to create bytecode, you're welcome to do so as long as you follow the rules of the JVM. Those are publicly available, but I don't know where.
You would have the same problem in C#, no? I'm not familiar with its reflection.emit package but it's quite possible that there's a Java equivalent that doesn't require on-the-fly compiling. Do you have an actual problem to solve or are you just comparison shopping? 
I do have an actual problem to solve. I will try to explain it. At buildtime I would like to get some database information about column lengths that I can then assign to UI field properties so that they validation lengths for these fields can be the same as the column lengths. I do not want to get the overhead of reading this at runtime. So I was of the opinion that at build time I could read the column length, build a map for this and the UI would be able to read this map at runtime to set its widget lengths appropriately. 
Now I'm confused. First you were asking how to generate bytecodes at runtime (while the program is running). Now you're talking about generating bytecodes at build time (while the program is compiling).
If it's the latter you want, simply write a program that reads the database metadata and outputs Java source code for the class that does the field validation. Then, compile this Java source file into bytecodes. 
Or your program that runs at build time could simply write those constant values to a properties file, which your main program would read in as part of its initialization. A lot simpler than trying to compile a class, even one that only contains constants.
(Incidentally, I have not yet got around to solving that identical problem in the application I am working on. But my proposed solution is to inquire into the database for the relevant field lengths (once only) as part of the program's initialization. Then if I changed a field length, I wouldn't have to recompile or rebuild anything.)

Categories

Resources