JPA ManyToMany with Eclipselink & GSON circular reference causing a stack overflow after server restart

[Former Member]

I have an issue with a manytomany bi-directional JPA relationship that is causing a stack overflow error

when the server is restarted (if there is data in the 3 tables that comprise the manytomany relationship.)

While I have a total of 5 entities in my application, I believe the issue is only related to

the Users and Application entities which use use a bi-directional manytomany relationship.

I have included the JPA diagram containing all entities for completeness

Below are the 2 entities that I believe are causing my circular reference stack overflow:

Application 

User

An Application can have many developers

A User can develop many applications

Here is how they are mapped:

//Application Entity:

    @ManyToMany

    private List<Users> users

//User Entity

   @ManyToMany(mappedBy = "users", fetch = FetchType.LAZY)

     private List <Applications> applications;

This results in 3 tables being created in the database as shown below:

APPLICATIONS

APPLICATION_USERS <  Join table created by JPA when using a ManyToMany relationship.

USERS

The application works fine when it is first deployed with empty tables.

A user registers for an application and this creates a row in the APPLICATIONS table (if the Application does not exist)

A row is also created in the USERS table if the user does not exist and the join table APPLICATION_USERS

is populated with the ID FROM THE APPLICATION Table called APPLICATIONS_ID and

an ID from the USERS table called USERS_ID as shown above.

You can add as many applications or users as you wish and the application works perfectly.

I have verified that data is being loaded and persisted into the 3 tables exactly as expected

Here is an example of the data in the tables after a user registers an Application:

Here is the problem:

Whenever the server is stopped and restarted or when the application is re-deployed using create-tables

(vs drop-and-create-tables) (and data is present in the tables). Application gets a stack overflow at each entities toString() function.

I have run this in debug with a breakpoints on the Applications toString() function and on the Users toString() function and I can click resume and watch each toString() function get called over and over until the stack overflow results.

Here is the console log:

//Entity query being executed

[EL Fine]: 2014-01-21 14:48:44.383--ServerSession(1615948530)--Connection(49767657)--Thread(Thread[http-bio-8080-exec-9,5,main])--SELECT t1.ID, t1.APPIDENTIFIER, t1.DATECREATED, t1.DATEMODIFIED, t1.DEVICETYPE FROM APPLICATIONS_Users t0, APPLICATIONS t1 WHERE ((t0.users_ID = ?) AND (t1.ID = t0.applications_ID))

//second entity query is invoked

[EL Fine]: 2014-01-21 14:50:02.444--ServerSession(1615948530)--Connection(1871047709)--Thread(Thread[http-bio-8080-exec-9,5,main])--SELECT t1.ID, t1.DATECREATED, t1.DATEMODIFIED, t1.EMAIL, t1.FIRSTNAME, t1.FULLNAME, t1.LASTLOGINDATE, t1.LASTNAME, t1.USERNAME FROM APPLICATIONS_Users t0, Users t1 WHERE ((t0.applications_ID = ?) AND (t1.ID = t0.users_ID))

[EL Finest]: 2014-01-21 14:50:02.471--ServerSession(1615948530)--Connection(1601422824)--Thread(Thread[http-bio-8080-exec-9,5,main])--Connection released to connection pool [read].

1. java.lang.StackOverflowError

       at java.util.Vector.get(Vector.java:693)

       at java.util.AbstractList$Itr.next(AbstractList.java:345)

       at java.util.AbstractCollection.toString(AbstractCollection.java:421)

       at java.util.Vector.toString(Vector.java:940)

       at org.eclipse.persistence.indirection.IndirectList.toString(IndirectList.java:797)

       at java.lang.String.valueOf(String.java:2826)

       at java.lang.StringBuilder.append(StringBuilder.java:115)

       at com.sap.crashlogserver.dao.entities.Applications.toString(Applications.java:150)

       at java.lang.String.valueOf(String.java:2826)

       at java.lang.StringBuilder.append(StringBuilder.java:115)

       at java.util.AbstractCollection.toString(AbstractCollection.java:422)

       at java.util.Vector.toString(Vector.java:940)

       at org.eclipse.persistence.indirection.IndirectList.toString(IndirectList.java:797)

       at java.lang.String.valueOf(String.java:2826)

       at java.lang.StringBuilder.append(StringBuilder.java:115)

       at com.sap.crashlogserver.dao.entities.Users.toString(Users.java:168)

       at java.lang.String.valueOf(String.java:2826)

       at java.lang.StringBuilder.append(StringBuilder.java:115)

       at java.util.AbstractCollection.toString(AbstractCollection.java:422)

       at java.util.Vector.toString(Vector.java:940)

       at org.eclipse.persistence.indirection.IndirectList.toString(IndirectList.java:797)

       at java.lang.String.valueOf(String.java:2826)

       at java.lang.StringBuilder.append(StringBuilder.java:115)

       at com.sap.crashlogserver.dao.entities.Applications.toString(Applications.java:150)

       at java.lang.String.valueOf(String.java:2826)

       at java.lang.StringBuilder.append(StringBuilder.java:115)

       at java.util.AbstractCollection.toString(AbstractCollection.java:422)

       at java.util.Vector.toString(Vector.java:940)

Based on a number of threads I have read I tried:

1. reversing the mappings,
2. Adding @JsonIgnoreputting to some of the entity fields
3. using fetch = FetchType.LAZY

and many other config tweaks but none of them resolved this issue.

Some of the suggestions like using transient fields I am not sure are supported in the JPA implemenation of eclipselink.

I also read a thread said that I should implement gson.ExclusionStrategy but I have not tried this yet.

I am new to Java and JPA and greatly appreciate any suggestions you may have to help me resolve this issue.

[Former Member · Accepted solution]

The issue we had with circular References with the Crashlog Reproted Server application has been resolved by my colleague Chris Chalmers. His modification to resolve the issue is detailed below and should be helpful to others who have a similar problem. I want to thank Johannes Staehlin for the information he provide to us that was of great assistance in resolving this issue.

From Chris:

"The issue with the CrashLog server application is caused by a
circular reference when serializing the web service response to JSON.
This circular reference occurs because the Application entity has a list
of users and the User entity has a list of Applications.  This
many-to-many relationship is causing the circular reference.  There is
nothing wrong with the current entity mappings or JPA annotations in terms of
which entity is the owning entity in this relationship.

The CrashLog server application uses Gson to serialize content to
JSON.  Our implementation of Gson contains an exclusion strategy whereby
properties can be ignored during the serialization process.  The
properties to be ignored need to be annotated with @JsonIgnore. 

So, to resolve the issue I did the following.

1.  Add the @JsonIgnore annotation to the 'applications' property of the User entity.

    @ManyToMany(mappedBy = "users")

    @JsonIgnore

     private List <Application> applications;

1. 2. Remove the
   'setSkipDataProtection(true)' call from the web service method causing the
   issue.  Telling Gson to skip data protection will result in all fields
   being serialized and the exclusion strategy to be ignored.  This will
   result in a circular reference.

      @GET

      @Path("/allusers")

      @Produces(MediaType.APPLICATION_JSON)

    public List<User> getAllUsers() {

//      
GsonMessageBodyHandler.setSkipDataProtection(true);

        List<User>
        users = new UserDAO().getAllDesc();

      return users;

I have also removed the
'setSkipDataProtection(true)' from other service methods where applicable and
updated the toString() methods on the entity objects as Johannes recommended.

Chris

[johannes_staehlin]

Hello Mark,

I think you problem is not directly related to JPA, but more a logical problem:
Some coding (probably a logger method) is calling the toString() method of an application object. Since you would like to append the toString() represenation of the user list, the toString() method of List is called. This method calls by default user.toString() on _every_ object in the list. These users call toString() on every application in their application list,...Since the user/application object has a bi-directional binding (every application is in the applicationList of it's userList elements), you end up in an endless loop.

I would recommend simplifying your toString() method by not printing out every detail of the entity. Anyway toStirng() is only for debugging and should be "concise but informative"

One example:

  @Override

  public String toString() {

       StringBuilder sb = new StringBuilder();

       sb.append("Application = [");

       sb.append(" id = " + id);

       sb.append(" user = ");

       for (User user : userList) {

            sb.append(user.getId());

            sb.append(", ");

       }

       sb.append("]");

       return sb.toString();

  }

Or let's just do it the other way around: What exactly do you expect as output of user.toString() and application.toString() ?

[Former Member]

Hi Johannes,

Below is what this toString() Method looks like in the application entity.

I think this method is required to serialize /  build the JSON result set that is called by  the rest service and passed back to the SAPUI5 application do you disagree ?

If so how would you modify this  ?

I am thinking it is just getting called over and over and I don't know how to stop it once it has built the expected object just once.

Regards,

Mark

String toString() {

return "Application [id=" + id + ", appIdentifier=" + appIdentifier + ", deviceType=" + deviceType + ", users=" + users

", crashLogs=" + crashLogs + "]";

}

[johannes_staehlin]

Hi Mark,

No. You would never ever use the toString() method for converting your objects. There a several better options. You could use JSON-Libraries such as Google Gson or probably even JAXB, that converts your Java objects directly to json using Annotations.

By the way I wrote my bachelor thesis ("Mobile meets Cloud: Creating Mobile APIs on a Java Cloud Platform") last year about this topic.

This also deals with this specific problem (endless loop during serialization to JSON)

So if you're more interested REST web services on HANA Cloud and JAX-RS/JAXB I can send it to you and probably point you to some of our internal example projects.

Best Regards, Johannes

[Former Member]

This might sound obvious, but could it be that toString() gets called in a recursive loop due to the many-to-many relationship between Application and User causing a stackoverflow?

[Former Member]

Hi Sanket,

What you say is true I believe but how to stop the stack overflow ?

I think this method is used to serialize the data into a JSON object that is consumed by the SAPUI5 front end application.

So I can't stop it from executing thru the normal building JSON object or I would not get any data back from the rest call.

Somehow I need to tell it so stop after it has built the object required once and not keep trying to build it over and over.

Is that a correct assumption ?

Regards,

Mark