javax.jcr.PathNotFoundException

JCR Data Model

http://www.ibm.com/developerworks/java/library/j-jcr/nodesproperties.gif

Repository -> Workspace -> Item (Node/Property)

Why PathNotFoundException?
From javadoc: Exception thrown when no Item exists at the specified path or when the specified path implies intermediary Nodes that do not exist.

From JCR troubleshooting topic: troubleshooting common exceptions: Within a workspace, each node is identified by its path and its uuid. In order to retrieve a node you must specify either its path or its uuid. The retrieval methods within the repository throw the exceptions "PathNotFoundException" and "ItemNotFoundException" to indicate that the node or property that you attempted to retrieve (either by path or uuid) does not exist in the current workspace.

How?
Handle it property - it is actually a warning.

cannot open '/etc/mail/local-host-names': World writable directory

What is World-writable?
A world-writable file or directory is one that anyone can alter. For directories this means that files can be created or deleted within it by anyone. Being world-writable is almost always a security risk.

Problem:
Get below error when try to start sendmail (/etc/init.d/sendmail start)
/etc/mail/sendmail.cf: line 94: fileclass: cannot open '/etc/mail/local-host-names': World writable directory

Solution:

1. Grant w to /etc, /etc/mail, /etc/mail/* => not work
2. Manually edit file sendmail.cf

replace line Fw/etc/mail/local-host-names with line
Fw-o /etc/mail/local-host-names

identically (if you got the same error for trusted-users)
replace line Ft/etc/mail/trusted-users with line
Ft-o /etc/mail/trusted-users

Shell script to monitor disk space

Motivation:
Space full caused db transaction hang issue last week, and this week it caused javax.jms.JMSException: No space left on device, herein ActiveMQ cannot write message to its queue. I decide to write a shell script to watch the disk space.

Steps:

1. Google for similar script and info
2. Write the shell script 
3. Start sendmail for email alert
4. Add script to cronjob for auto monitoring

Commands:

1. /etc/init.d/sendmail => Usage: /etc/init.d/sendmail {start|stop|restart|condrestart|status}
2. /etc/init.d/crond => Usage: /etc/init.d/crond {start|stop|status|reload|restart|condrestart}

Two options to add script to cronjob:

1. crontab -e
2. Put shell script to /etc/cron.* folder (cron.daily/   cron.hourly/  cron.monthly/ cron.weekly/)

Script:

# http://www.cyberciti.biz/tips/shell-script-to-watch-the-disk-space.html
# set admin email for alert
ADMIN_EMAIL="admin@sample.com"
# set alert level
ALERT_LEVEL=90
# set log folder to get deleted
LOG_FOLDER=/opt/logs/*

df -HP | grep -vE '^Filesystem|tmpfs|cdrom' | awk '{ print $5 " " $1 }' | while read output;
do
  echo $output
  used=$(echo $output | awk '{ print $1}' | cut -d'%' -f1 )
  partition=$(echo $output | awk '{ print $2 }' )
  if [ $used -ge $ALERT_LEVEL ]; then
    echo "Running out of space \"$partition ($used%)\" on $(hostname) as on $(date)" |
    mail -s "Alert: Running out of disk space - $used% used" $ADMIN_EMAIL

              echo "Purging log files ..."
              find $LOG_FOLDER -mtime +2 -exec rm -f {} \;

              echo "DONE"   

  fi
done

Why use df - HP?

linux01:root > df -H
Filesystem             Size   Used  Avail Use% Mounted on
/dev/sda2              8.0G   2.7G   5.0G  35% /
192.168.253.131:/u001/ReleaseArchive/
                       289G   203G    78G  73% /jdrive

linux01:root > df -H -t ext3
Filesystem             Size   Used  Avail Use% Mounted on
/dev/sda2              8.0G   2.7G   5.0G  35% /

linux01:root > df -HP
Filesystem             Size   Used  Avail Use% Mounted on
/dev/sda2              8.0G   2.7G   5.0G  35% /
192.168.253.131:/u001/ReleaseArchive/   289G   203G    78G  73% /jdrive

Reference:
http://www.cyberciti.biz/tips/shell-script-to-watch-the-disk-space.html

Emma build caused SecurityException

Problem:
Web application throws below exception:

java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
at sun.security.util.SignatureFileVerifier.processImpl(SignatureFileVerifier.java:221)
at sun.security.util.SignatureFileVerifier.process(SignatureFileVerifier.java:176)
at java.util.jar.JarVerifier.processEntry(JarVerifier.java:277)
at java.util.jar.JarVerifier.update(JarVerifier.java:188)
at java.util.jar.JarFile.initializeVerifier(JarFile.java:321)
at java.util.jar.JarFile.getInputStream(JarFile.java:386)
at org.apache.catalina.loader.WebappClassLoader.findResourceInternal(WebappClassLoader.java:2018)
at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:1786)
at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:890)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1354)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1233)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)

Solution:
It was caused by emma build, not sure of the exact root cause, but using standard (non-emma) build will not have this issue.

Notes:

1. Usually the exception is complaining the signed archive is corrupted
2. Emma is a open source java code coverage tool http://emma.sourceforge.net/

Tomcat is not responding due to ORA-00257

Symptom:
Web team reports that request to tomcat is not responding all of a sudden. The environment is an isolated one with access control, which means there is no system or application change recently.

Application Log:
Caused by: com.inet.ora.Ora4SQLException: [Oracle] #1 ORA-00257: archiver error. Connect internal only, until freed.
[Oracle] #1
    at com.inet.ora.i.a(Unknown Source)
    at com.inet.ora.OraConnection.a(Unknown Source)
    at com.inet.ora.OraConnection.q(Unknown Source)
    at com.inet.ora.OraConnection.<init>(Unknown Source)
    at com.inet.ora.l.<init>(Unknown Source)
    at com.inet.ora.i.a(Unknown Source)
    at com.inet.ora.OraDriver.connect(Unknown Source)
    at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
    at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:294)
    at org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1247)
    at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1221)
    ... 46 more

Thread Dump:
"http-8080-12" daemon prio=10 tid=0x0000000046b20c00 nid=0x18e0 runnable [0x000000004954e000..0x000000004954fb90]
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:129)
    at com.inet.ora.OraConnection.i(Unknown Source)
    at com.inet.ora.OraConnection.a(Unknown Source)
    - locked <0x00002aab89aad9d0> (a com.inet.ora.l)
    at com.inet.ora.ax.a(Unknown Source)
    at com.inet.ora.ax.c(Unknown Source)
    at com.inet.ora.ax.execute(Unknown Source)
    at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:169)
    at com.ibatis.sqlmap.engine.execution.SqlExecutor.executeUpdate(SqlExecutor.java:81)
    at com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.sqlExecuteUpdate(GeneralStatement.java:200)
    at com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.executeUpdate(GeneralStatement.java:78)
    at com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.update(SqlMapExecutorDelegate.java:505)
    at com.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.update(SqlMapSessionImpl.java:90)
    at com.ibatis.sqlmap.engine.impl.SqlMapClientImpl.update(SqlMapClientImpl.java:67)

Root Cause:
Oracle archived redo logs used up disk space. Linux 'df' shows the space is 100% used.

Solution:
As the log mentioned, freed space. We did a quick move of archived log files to another disk for the fix. Meanwhile, we are discussing 2 options: either turn on NOARCHIVELOG, or write a script to do automatic gzip and move to other disk.

Eclipse: Class file name must end with .class

Problem:
Today after I switched to another workspace to work on another CVS branch. I got error in Java search "Class file name must end with .class" when I type Ctrl+Shift+G (for reference in workspace).

Options I tried:

1. Clean all projects and rebuild workspace (not work)
2. Close the projects and reopen again (not work)
3. Close Eclipse and start again (not work)
4. Add -clean in Eclipse startup script (e.g. C:\Workshop\eclipse\eclipse.exe  -clean), and this worked!!

Another option I didn't try:
     a.  Close Eclipse
     b.  Delete <workspace>/.metadata/.plugins/org.eclipse.jdt.core/*.index
     c.  Delete <workspace>/.metadata/.plugins/org.eclipse.jdt.core/savedIndexNames.txt
     d.  Start Eclipse again 

See http://stackoverflow.com/questions/6816/eclipse-class-file-name-must-end-with-class-exception-in-java-search

Root Cause:
Unknown, but mostly like it is Eclipse cache issue. My Eclipse version is as below

Eclipse Platform
Version: 3.4.1
Build id: M20080911-1700

Oracle redo log slows down application

Symptom:
J2EE application on tomcat server suddenly has high latency, and has very slow server side processing response, and all 150 threads are used up. Regular request even takes more than 30 seconds while in normal case it takes around 100ms.

Production Info:

1. SW: Oracle 10g RAC, Tomcat6.0, JDK1.6, CentOS4.4
2. No production outage or HA failover/failback
3. No stress test or peak load

Root Cause:
NFS mount point hung which in turn slowed the archiving the logs to the NFS mount point, so the redos were not getting archived fast enough, and caused the latency.

Suppress Findbugs warnings

Background:
Last week one colleague sent me an email to confirm a false warning from Findbugs. He explained it was SpringFramework init method, so need "Write to static field from instance method". As we know, it is a bad practice if multiple instances are manipulated. But here we need suppress the warning.

    public void setApplicationContext(ApplicationContext applicationContext) {
        context = (ApplicationContext) applicationContext;
        getServiceManager().getStatus().setSource(getServiceManagerUrl());
    }

Findbugs, PMD, Checkstyle, JTest and etc are popular static java code analysis tools. Findbugs uses bug detectors (pluggable) to analyze java bytecode (compiled class files) based on certain bug patterns. It self uses BCEL, ASM and dom4j etc open source libraries.

Installation:
Install Findbugs to Eclipse IDE is straightforward, and same as other Eclipse plugin. (Help -> Software Updates... -> Available Software -> Add Site using http://findbugs.cs.umd.edu/eclipse/). I use Findbugs-1.3.9 and JDK1.6 in the test of suppress code.

Run Findbugs:
After installation and restart Eclipse, in Package Explorer, right-click to find "Find Bugs" menu. We can configure to run Findbugs automatically.

Suppress Findbugs warning:
Unlike PMD leverages the @java.lang.SuppressWarnings annotation, or JTest uses predefined comments, Findbugs defines its own annotation in edu.umd.cs.findbugs package, so we need use below annotation

    @edu.umd.cs.findbugs.annotations.SuppressWarnings(value =
        "ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD", justification = "Spring init method")
    public void setApplicationContext(ApplicationContext applicationContext) {
        context = (ApplicationContext) applicationContext;
        getServiceManager().getStatus().setSource(getServiceManagerUrl());
    }

or

    import edu.umd.cs.findbugs.annotations.SuppressWarnings
    @SuppressWarnings(value =
        "ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD", justification = "Spring init method")
    public void setApplicationContext(ApplicationContext applicationContext) {
        context = (ApplicationContext) applicationContext;
        getServiceManager().getStatus().setSource(getServiceManagerUrl());
    }

Reference:
http://findbugs.sourceforge.net/manual/eclipse.html
http://findbugs.sourceforge.net/manual/annotations.html

Redirects handling in httpclient

Problem:
Httpclient cannot get correct response from production server (http://production.domain.com/servlet.do)

Root cause:
In production, http request will be redirected to https by load balancer (302 redirect code). There are a few types of redirect that HttpClient can't handle automatically, including 302, in entity enclosing methods (POST and PUT).

    * 301 Moved Permanently. HttpStatus.SC_MOVED_PERMANENTLY
    * 302 Moved Temporarily. HttpStatus.SC_MOVED_TEMPORARILY
    * 303 See Other. HttpStatus.SC_SEE_OTHER
    * 307 Temporary Redirect. HttpStatus.SC_TEMPORARY_REDIRECT

    * 300 Multiple Choices. HttpStatus.SC_MULTIPLE_CHOICES
    * 304 Not Modified. HttpStatus.SC_NOT_MODIFIED
    * 305 Use Proxy. HttpStatus.SC_USE_PROXY

Solution:
Handling redirects manually in codes for entity enclosing methods.

Sample Code:

    // Create an instance of HttpClient.

    HttpClient client = new HttpClient();

   

    // Create a method instance.

    //HttpMethod method = new GetMethod(url); //good with redirects unless setFollowRedirects(false)

    //method.setFollowRedirects(true);

    PostMethod method = new PostMethod(url); //entity enclosing methods

   

    // Provide custom retry handler is necessary

    method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,

            new DefaultHttpMethodRetryHandler(3, false));

    try {

      // Execute the method.

      int statusCode = client.executeMethod(method);

     

      if (statusCode != HttpStatus.SC_OK) {

          Header locationHeader = method.getResponseHeader("location");

          method.releaseConnection();

         

          if (locationHeader != null) {

              System.out.println(locationHeader.getValue());

              method = new PostMethod(locationHeader.getValue());

              statusCode = client.executeMethod(method);

          }

      }

      // Read the response body.

      byte[] responseBody = method.getResponseBody();

      // Deal with the response.

      // Use caution: ensure correct character encoding and is not binary data

      System.out.println(new String(responseBody));

    } catch (HttpException e) {

      System.err.println("Fatal protocol violation: " + e.getMessage());

    } catch (IOException e) {

      System.err.println("Fatal transport error: " + e.getMessage());

    } finally {

      // Release the connection.

      method.releaseConnection();

    } 

Notes:

1. Http specification defines two entity enclosing methods, POST and PUT.
2. Redirects is automatically handled by httpclient in GET method, unless setFollowRedirects(false).

Background:
Httpclient is a popular http1.1 compliant agent implementation, open sourced at apache. There are 2 typical versions now, one is legacy commons httpclient (v3.1), and the other is the new HttpComponents (v4.x), successor of and replacement of commons-httpclient.

commons-httpclient-3.1.jar (org.apache.commons.httpclient)
commons-codec-1.4.jar (org.apache.commons.codec)

httpclient-4.0.1.jar (org.apache.http)
httpcore-4.0.1.jar (org.apache.http)

Reference:
http://hc.apache.org/
http://hc.apache.org/httpclient-3.x/redirects.html
http://hc.apache.org/httpcomponents-client-4.0.1/tutorial/html/index.html

Eclipse is running in a JRE, but a JDK is required

Problem:
After I install m2eclipse plugin, every time I launch eclipse IDE, I get an alert saying "Eclipse is running in a JRE, but a JDK is required".

Root Cause:
Maven needs some jar files from JDK distribution.

Solution:
Add -vm  option to eclipse.ini before -vmargs option.

Here is copy of my eclipse.ini (Windows OS, Eclipse 3.4.1)

-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
256M
-framework
plugins\org.eclipse.osgi_3.4.2.R34x_v20080826-1230.jar
-vm
C:\Program Files\Java\jdk1.6.0_20\jre\bin\client\jvm.dll
-vmargs
-Dosgi.requiredJavaVersion=1.5
-Xms40m
-Xmx512m

Notes:

1. The -vm option must come before the -vmargs, since everything after -vmargs is passed to the JVM instead of Eclipse
2. The -vm option and its value (the path to the JVM) must be on separate lines.
3. The path to the JVM can be to java.exe, javaw.exe or jvm.dll per Eclipse wiki page, but my test shows jvm.dll works.
4. If the path with white space (e.g. C:\Program Files) cannot work, double quote the path.
5. I also set -vmargs option in Eclipse start up command line (e.g. C:\Workshop\eclipse\eclipse.exe  -clean -showlocation WAPI2.0 -refresh -vmargs -Xms256M -Xmx384M)

Reference:
http://wiki.eclipse.org/Eclipse.ini