Oracle NoSQL & Hadoop - NoSQL Database

Hi all,
Did some one experiment Oracle NoSQL with Hadoop want to share the results?
Thanks 

user962305 wrote:
Hi all,
Did some one experiment Oracle NoSQL with Hadoop want to share the results?This is a very broad question and therefore difficult to answer. Is there anything in particular you are asking about in terms of Hadoop and Oracle NoSQL Database?
Oracle NoSQL Database is packaged with a Hadoop Input Stream. See the doc at:
http://docs.oracle.com/cd/NOSQL/html/javadoc/oracle/kv/hadoop/package-summary.html
There is also an example referenced in that javadoc.
Charles Lamb 

Did Oracle realize a business test with map/reduce?
To see how the usage of map/reduce with Oracle NoSQL database will improve performance on some jobs. 

user962305 wrote:
Did Oracle realize a business test with map/reduce?
To see how the usage of map/reduce with Oracle NoSQL database will improve performance on some jobs.No. 

Oracle NoSQL Database includes Hadoop based example \kv-1.2.123\examples\hadoop\CountMinorKeys.java.
To test the example:
1. Add the Hadoop related Jar files from the \hadoop-0.22.0\lib directory to classpath.
2. Configure a Hadoop Cluster
3. Start the Hadoop Cluster
4. Run a Map/Reduce Job
Edited by: dvohra16 on Apr 7, 2012 11:16 AM 

I'm wondering if the original question was trying to understand whether Map Reduce runs on NoSQL DB. E.g. NoSQL is a different storage under MapReduce. To ensure no confusion remains: you do not run Map Reduce on the NoSQL DB storage infrastucture. You use an input format and get data out of NoSQL and process on Hadoop...
JP 

To see how the usage of map/reduce with Oracle NoSQL database
Map/Reduce runs on Hadoop with Oracle NoSQL database data. 

Thanks Charles
Is there anything you suggest at this point to get around this issue?
Based on the information found on the link below, the issue seems to have been fixed on Map/Reduce 0.21.0
https://issues.apache.org/jira/browse/MAPREDUCE-954
I can either wait till Cludera release next CDH version (hopefully will include the fix) or try to apply the patch that is attached to MAPREDUCE-954 bug however It is not 100% clear to me how to apply it
Any detailed instructions how to apply this patch would be much appreciated. 

Thans Charles
Is there anything you might suggest at this stage to get around this issue?
According to the link below, the bug fix has been included in Map/Reduce 0.21.0. Unfortunately the lastest CHD 4.1.2 includes Map/Reduce 0.20.0.
https://issues.apache.org/jira/browse/MAPREDUCE-954
I can either wait for the next CDH version (hopefully will include the fix) or try to apply the patch that is attached to MAPREDUCE-954 link above.
However it is not entire clear to me how to apply the patch.
Any detailed instruction how to apply to make the "CountMinorKeys" sample work, would be appreciated.
Cheers 

Thans Charles
Is there anything you might suggest at this stage to get around this issue?
According to the link below, the bug fix has been included in Map/Reduce 0.21.0. Unfortunately the lastest CHD 4.1.2 includes Map/Reduce 0.20.0.
https://issues.apache.org/jira/browse/MAPREDUCE-954
I can either wait for the next CDH version (hopefully will include the fix) or try to apply the patch that is attached to MAPREDUCE-954 link above.
However it is not entire clear to me how to apply the patch.
Any detailed instruction how to apply to make the "CountMinorKeys" sample work, would be appreciated.
Cheers 

Thans Charles
Is there anything you might suggest at this stage to get around this issue?
According to the link below, the bug fix has been included in Map/Reduce 0.21.0. Unfortunately the lastest CHD 4.1.2 includes Map/Reduce 0.20.0.
https://issues.apache.org/jira/browse/MAPREDUCE-954
I can either wait for the next CDH version (hopefully will include the fix) or try to apply the patch that is attached to MAPREDUCE-954 link above.
However it is not entire clear to me how to apply the patch.
Any detailed instruction how to apply to make the "CountMinorKeys" sample work, would be appreciated.
Cheers 

Thans Charles
Is there anything you might suggest at this stage to get around this issue?
According to the link below, the bug fix has been included in Map/Reduce 0.21.0. Unfortunately the lastest CHD 4.1.2 includes Map/Reduce 0.20.0.
https://issues.apache.org/jira/browse/MAPREDUCE-954
I can either wait for the next CDH version (hopefully will include the fix) or try to apply the patch that is attached to MAPREDUCE-954 link above.
However it is not entire clear to me how to apply the patch.
Any detailed instruction how to apply to make the "CountMinorKeys" sample work, would be appreciated.
Cheers 

Thans Charles
Is there anything you might suggest at this stage to get around this issue?
According to the link below, the bug fix has been included in Map/Reduce 0.21.0. Unfortunately the lastest CHD 4.1.2 includes Map/Reduce 0.20.0.
https://issues.apache.org/jira/browse/MAPREDUCE-954
I can either wait for the next CDH version (hopefully will include the fix) or try to apply the patch that is attached to MAPREDUCE-954 link above.
However it is not entire clear to me how to apply the patch.
Any detailed instruction how to apply to make the "CountMinorKeys" sample work, would be appreciated.
Cheers 

Thans Charles
Is there anything you might suggest at this stage to get around this issue?
According to the link below, the bug fix has been included in Map/Reduce 0.21.0. Unfortunately the lastest CHD 4.1.2 includes Map/Reduce 0.20.0.
https://issues.apache.org/jira/browse/MAPREDUCE-954
I can either wait for the next CDH version (hopefully will include the fix) or try to apply the patch that is attached to MAPREDUCE-954 link above.
However it is not entire clear to me how to apply the patch.
Any detailed instruction how to apply to make the "CountMinorKeys" sample work, would be appreciated.
Cheers

Related

Java ClassLoader Issue - JDK 1.5

Hello Everyone,
We've recently been experiencing some issues with some of our OAS application servers timing out daily. The goal of this post is to try and understand what is causing these issues, and in addition, if there is any solutions we can up-take without doing any major upgrades. (JDK versions)
Problem Overview:
We are currently using 3 application servers (nodes) that are running Jive's Clearspace Community platform in a clustered environment. The problem is that on a daily basis, one of these nodes will end up timing out and become unresponsive, and then needs to be restated to start behaving correctly again with the other 2 nodes.
Working with Jive, we think we have narrowed the problem to a class-loader issue that occurs every-time one of these time outs occur, that is external to Clearspace Community. Doing a bit of research, we found a document that may isolate this problem to the current version of JDK we are using with Clearspace Community (JDK 1.5) which I highlight more in the next section.
Log Files:
Looking into the log files, we've consistently seen the following error every time we have a timeout on any of the nodes:
Found one Java-level deadlock:
=============================
"AJPRequestHandler-HTTPThreadGroup-16":
  waiting to lock monitor 0x08232a3c (object 0xa8c17ee8, a oracle.classloader.PolicyClassLoader),
  which is held by "AJPRequestHandler-HTTPThreadGroup-29"
"AJPRequestHandler-HTTPThreadGroup-29":
  waiting to lock monitor 0x9035585c (object 0xaa0c49c0, a java.net.URLClassLoader),
  which is held by "AJPRequestHandler-HTTPThreadGroup-16"
Java stack information for the threads listed above:
===================================================
"AJPRequestHandler-HTTPThreadGroup-16":
        at oracle.classloader.PolicyClassLoader.internalLoadClass(PolicyClassLoader.java:1659)
        - waiting to lock <0xa8c17ee8> (a oracle.classloader.PolicyClassLoader)
        at oracle.classloader.PolicyClassLoader.loadClass(PolicyClassLoader.java:1635)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:299)Working with Jive we found the following documentation on the Oracle site that identified a similar issue:
Here
Jive's Recommendation:
Jive has recommended to upgrade our JDK from version 1.5 to 1.6 (via the draft proposal above) to resolve the issue, however at this time we cannot do that easily. I am trying to identify if there is more of a "band-aid" solution that would allow us to retain using JDK 1.5 for the short-term. (We have plans to upgrade our JDK and Jive platform to the latest version later this summer, so we are just trying to get by at this point without having our admins deal with these daily time out issues.)
Conclusion:
So I wanted to post to the Oracle / Java community and see if anyone possibly had any knowledge into this problem? In addition, if there is a viable solution WITHOUT having to upgrade our JDK to version 1.6, that is most likely the avenue we are trying to explore at this point.
If anyone needs any additional information or clarifications, please do not hesitate to ask, and I'll help as best as I can.
Thank you in advance for all of your time and help, it is greatly appreciated!
--Asif Ahmad                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            
This looks very much like some sort of thread unsafety in oracle.classloader.PolicyClassLoader. I'm not really sure how upgrading the VM would help with that.
Do you have a Java bug id? 
Hey cotton.m,
Can you give me some information on the PolicyClassLoader and the exact?
In addition, where do I locate the Java bug id? I was not sure where to look for it.
Thanks again!
--Asif                                                                                                                                                                                                                                                                                                                                                                                                            
A4orce84 wrote:
Hey cotton.m,
Can you give me some information on the PolicyClassLoader and the exact?Errr. no. I am just reading the stacktrace and dump. It appears that there is a custom class loader provided by Oracle for something that you are using. And that custom class loader is having a deadlock condition.
In addition, where do I locate the Java bug id? I was not sure where to look for it.
Thanks again!No I don't have a bug id. It's your bug.
To try and help you out. You've described a problem, a deadlocking custom class loader. Okay. You say that Jive and Oracle both identified that issue as being in Java. Now to be honest my guess is that they're wrong. Jive for example is pretty clueless IMO. But regardless if what they say is true, that this is a Java bug that is fixed in 1.6 then they surely have an id for the Java bug database.
And that information will help because it might expand on what exactly are the steps to re-create the problem and how you might workaround it. And if they don't have an answer for that then it's a bug in their code.
In neither case can we really help you here but if I were you my first step would be to get nailed down whose bug this is and why this is happening. 
Presuming there is actual code that you produced...
One or more of the following is true.
1. Bug in Oracle software (this is a Oracle JDBC driver presumably and not an Oracle VM.)
2. Bug in VM you are using
3. Bug in Jive Software
4. Bug in your software.
I am also guessing that you can't replicate it. And you are hoping to throw something at it to fix it.
Possible solutions
1. Replace one or all of the Oracle, VM or Jive software
2. Find the bug in your software
3. Determine where it occurs in your software and find another way to re-write it.
If and only if you can find some way to replicate it and you would probably need to localize it to one of the components above then you could write a bug against the particular vendor.

Why no "datapatch" for patching databases under docker

Hi everybody, I'm trying some stuff with the provided dockerfiles from here https://github.com/oracle/docker-images  to create docker images for Oracle Database Standard Edition 2. What I'm wondering is: I'm able to install Oracle patches and so on into the database home. But why there is no script for doing the "datapatch" step so that the patches will be installed / implemented into my CBD and PDB on that image? Thanks and regards,David
Hey David. The best way to get support for those images is to open an issue on GitHub in that repo. Or check out the existing issues as it may already be possible in some way.
That was what I've searched first... On github, they redirect to this forum 
Whoops. I'll fix that now. The best place is GitHub.

Oracle JET v5.2.0 Tooling Migration - documentation issues

Hello, I came across a small mistake in the documentation for the migration of JET tooling of an application, into version 5.2.0. In the section regarding migrating from 5.x.x to 5.2.0, there is a typo in the following npm command, which causes it to fail: npm install #oracle/oraclejet#~5.2.0 #oracle/oraclejet-tooling~#5.2.0 --save where it should be instead: npm install #oracle/oraclejet#~5.2.0 #oracle/oraclejet-tooling#~5.2.0 --save  In the previous section regarding migrating from a previous major release (4.x.0 to 5.2.0), there is a similar command, but just lacking the '~' (tilde) before the version number, like so: npm install #oracle/oraclejet#~5.2.0 #oracle/oraclejet-tooling#5.2.0 --save Although that is not as bad of an issue since it will still install the proper package, just being maybe a bit too specific about it's version.  Cheers,José   
Thanks for pointing this out Jose. We'll get it corrected as soon as possible.

Worklist Manager SE

We are working on a project based on OpenESB with NetBeans as IDE and one of the main goals is to involve human interaction in processes using some sort of specialized component (so not just by invoking a receive activity from a user interface). We've discovered that there is already a JBI component which can do exactly what we want - the Worklist Manager SE.
I was expecting the developement tool for this module to be officialy included with the recent 6.0 release of NetBeans, but it's not. I've tried to install the nbm files containing the Worklist Project plugin to NetBeans, but haven't found a way to do it since there was some *"module missing error" for xmlbeans*. I've downloaded the lastest OpenESB installer but it also doesn't work properly. The web application included is missing a package com.sun.workflow.client which I can't find anywhere and more importantly, when I rebuild the worklist project, the xform file is not correctly generated (there are no fields, just the Submit button).
I don't know if the JBI components developers are watching this forum, but it would be very useful for us if we could get in touch with someone who is working on the WLM SE module to learn more about the current state of this component. Perhaps we could also help to improve it during the developement of our project.
If someone is using or has tried to use this component with success I'd be very thankful if he can provide information about some functional version configuration for NetBeans & OpenESB. Best would be to find out how to develop a Worklist Application on the recent NetBeans 6.0 release.
Thanks anyone for answers, any help is very appreciated. 
Hello,
Unfortunately, Worklist Manager SE project is suspended (temporarily) because lack of resource, but when resource is available in the near future, it or a similar product will continue. It had been under active development for a year or so, but did not make into a release.
If you or your team is interested in doing some work in this area, please let us know.
Thanks
Mei 
Hi,
I've tried running the sample with latest openESB and the worklist web app has errors, I also tried looking in the sources and I figured the error comes from:
com.sun.workflow.client API which I can't find in the CVS.
is there any news on the WLM SE, is it back in development, why is there no indication that work on this has been stopped in the wiki.
cheers,
Dror 
I have got the same issue and want to know if the package com.sun.workflow.client.* is available now?
Can anyone provide current status on this or future available date? 
The samples are working, please see: http://wiki.open-esb.java.net/Wiki.jsp?page=WLMSE and http://wiki.open-esb.java.net/attach/WLMSE/WLMSE_JAVAONE_2007_SAMPLE_INSTRUCTION.pdf.
However, I suggest you wait till next week to try the instruction because I'm fixing some glitches and in the process of making the deployment easier.
BTW, please see project plan:
http://wiki.open-esb.java.net/Wiki.jsp?page=WLMProjectPlan 
Are the fixes that have been shown in the project plan for the WLM available for download?
I am wanting to use the WLM samples with Netbeans 6.5 daily builds (latest), and am having trouble getting them to work.
please advise.
regards,
peter. 
Hi, Peter,
The samples should all work per instruction doc, have you tried them? What problems do you have? Note that you must follow the current instruction doc (i.e. if you download the doc a month ago and tried to follow that with the new binaries, it might not work since the document is updated from time to time).
BTW, project plan is followed > 90%, so I suppose the fixes you mean are the web app fixes, they should be fixed.
http://wiki.open-esb.java.net/attach/WLMSE/WLMSE_JAVAONE_2007_SAMPLE_INSTRUCTION.pdf
Thanks
Mei 
Hi Mei,
thank you for responding.
I will try it again, using the latest version of that doc you pointed to, and, if I encounter issues, will document them here for you.
btw: the doc you point to - does it have a version, last update date and or change log within it that identifies which version of the doc I already have versus the one that's on the web site? it would really be helpful to know that this is a living doc and which areas have changed.
regards,
peter. 
Mei,
I just saw in the version of the doc that it still contains the warning about using not using JDK 6 due to an xform generation issue.
Looking at the project plan, I thought that this issue had been resolved.
is the version of the downloadable nbm's able to be used with JDK6 and the worklist task manager web app or not?
I really want to be able to use JDK6.
please advise.
regards,
peter. 
Hi, Peter,
The JDK6 issue is not resolved yet, the workaround is you use JDK5 to generate the xforms (build the project the first time and generate xhtml files) and then use JDK5 to do the rest of things.
The issue isn't so much in xform generation, but the xerces API packaged inside JDK6 is not backward compatible and I believe there is a solution down the track, I will move the issue to our immediate to-do list
Thanks
Mei 
Hi, Peter,
Good question.
I will add the version on the doc and change log in the future.
In the meantime, maybe you can see the version from attachment (http://wiki.open-esb.java.net/PageInfo.jsp?page=WLMSE/WLMSE_JAVAONE_2007_SAMPLE_INSTRUCTION.odt) for versions, but not the change log.
Thanks
Mei 
Hi Mei,
I'm sorry, did you mean to use JDK5 to generate the xforms/xhtml and then use JDK6 to do the rest?
the way you wrote the response it appears they should be different, but you call out JDK5 for the before and after.
regards,
peter. 
Hi, Peter,
Today, I checked in the fix for this issue and tested it on JDK 6, it should be working now.
The next openESB installer will include this fix.
For now, the sample app should have the xform generated, so you need JDK 6 only.
thanks
Mei 
hello Mei,
well, ok, previously, I tried using following the instructions but used a netbeans 6.5 daily build, and had problems.
this time, I am trying to follow the instructions more carefully, and I downloaded the openesb daily all in one build, and am trying to install it. (ie: openesb-installer-windows-20080929.exe)
upon execution of the install program, I see a dialog box saying that it could not find a suitable java JVM, and that I should use the --javahome option to specify the location of the jdk.  so I did.
I tried pointing it at the JDK6 install, and got an error dialog. I then installed (didn't have it installed until now) JDK5, but I still get the same dialog box mentioned earlier, where it shows:
-------------------------------------------------------------------------------------------------------------------
Unsupported Java VM version
The java VM at C:\Program Files\Java\jdk1.5.0_16 has the unsupported version
--------------------------------------------------------------------------------------------------------------------
I'm also using the --userdir option to keep the setup details distinct from any other netbeans install I have on the machine, which happens to be windows 2008 server x64.
note: as much as I'm pointing it at the x64 version of the jdk, I also have the 32bit jdk installed for JDK6, and it gave me the same sort of kickback.
please advise.
regards,
peter. 
Hi Mei,
thank you for making JDK6 usable... that's great!
if only I could install the all in one bundle. see my other post....
thank you for your efforts - it is appreciated!
regards,
peter.

Sun IDM Upgrade Help

Hi folks,
I am upgrading my Sun IM from 5x -> 6x ->7x version, can somebody tell me how to work for
custom xml files ? How to import my custom xmls ? Do i need import manually ? or through some script.
Anticipating quick help.
Randhir 
Hi Randhir:
Have you had a chance to look at the upgrade docs providede by Sun? Those were very helpful when we upgraded from 5 to 7 and now looking at an 8 upgrade. 
Randhir,
If you are upgrading from 5 -> 6 -> 7 or beyond, you'll probably want to get some expert help in this area either from Sun or from a partner. You need to consider custom resource adapters, workflows/forms/rules, etc. deprecated functionality, improved or new functionality that may replace previous areas of the product, etc.
If you're simply worried about importing your XML objects - you should make it a point to keep your customizations within a code repository of some sort (CVS, Subversion, etc.), not modify the original files that came with the software, and use the Build environment that's documented. Very helpful when tracking changes and differences.
Finally you can look into using the ImportCommand (look in the /samples directory for files using it) to include other XML files after importing just one. The trick is to do this from the command line though so the path referenced to the XML object is accurate. If you do it through the UI you'll need to change all the paths.
Good luck,
Damon 
Hi DamonL ,
Thanks for help , i will follow the Guideline from Sun.
Another small help from ...
Regarding schema upgrade, call you tell me commands / query which will update my schema for
IDM 5x to Higher Version.
Thanks
Randhir Singh 
Hi,
If you look at the db_scripts directory from each set of installation files it will provide you with the necessary scripts to upgrade the repository to the next version. 
Hi Chapo,
Thanks for help.
As most major releases and some minor releases of Identity Manager include "Database Table" changes. consequently, one might have to modify the sample SQL scripts for respective environment.
Can you please tell me exact commands / query how to update / modify the Script.
(I just hope i am clear with my query)
Thanks
Randhir Singh 
Hi Randhir,
Just check in douments provided by Sun for upgrade, it should include all the required steps for database repository changes.
THANKS 
Randhir,
Sun makes it a point, if at all possible, to not change the database schema between minor changes (7.1 -> 7.1.1, etc) but from 7.0 to 7.1 or 8.0 could possibly have schema changes.
I sometimes run multiple IDM environments on the same box for development, and have needed to modify the schema script from time to time. Typically I only replace the waveset tablename with another name, in VI I do a global search/replace and that's about it.
Bottom line: do this in a development environment and learn from your experience. I don't think any one person could tell you what you'll need to do unless they were onsite looking at your environment.
Best of luck!
Damon 
Randhir,
I don't think there should be any issue to upgrade your repository, because there is already scripts provided for upgrade by Sun. You can find it in product's db_script directory name like "upgradeto71from70.oracle". I had migrated the IdM 4.1 to 5.5 2 year back and did follow the same steps provided by product documantaion.
Thanks,
Amitesh 
Thanks Amitesh for help
Randhir SIngh 
Hi Amitesh,
Where can i find the script "db_script" for IDM 6x version ?
Thanks
Randhir Singh 
Randhir,
you can download the sun idm 6x version and within that product dir u will find the db_script folder.
Thanks 
hi guys,
i am upgrading from 5.5 to 8.1 and was wondering where i can find language packs for 6.0 and service packs for 6.0. do i need to apply service packs in case of a multi-hop upgrade?

Categories

Resources