[saxon] AssertionError when exceeding FeatureKeys.MAX_COMPILED_CLASSES in Saxon-EE 9.8

[Rademacher, Gunther]

In my testing environment, Java assertions are enabled by default. This caused Saxon-EE to throw an AssertionError, when the limit imposed by FeatureKeys.MAX_COMPILED_CLASSES is exceeded.

There is no such error in the production (server) environment, but I wonder how to set MAX_COMPILED_CLASSES. It looks to me like this is a hard limit, that prevents further bytecode generation once it has been reached. As I have no indication of generated classes not being garbage collected, I tend to set it to Integer.MAX_VALUE. Would that return me to 9.7 behaviour, to this respect?

Also, is an assertion adequate for observing this limit? After all it addresses a situation that may legally occur, so refusing to continue seems a bit harsh to me, even when assertions have been enabled.

Best regards
Gunther

Software AG – Sitz/Registered office: Uhlandstraße 12, 64297 Darmstadt, Germany – Registergericht/Commercial register: Darmstadt HRB 1562 - Vorstand/Management Board: Karl-Heinz Streibich (Vorsitzender/Chairman), Eric Duffaut, Dr. Wolfram Jost, Arnd Zinnhardt, Dr. Stefan Sigg; - Aufsichtsratsvorsitzender/Chairman of the Supervisory Board: Dr. Andreas Bereczky - http://www.softwareag.com

[O'Neil Delpratt]

Hi Gunther,

Details of the MAX_COMPILED_CLASSES is provided here: http://www.saxonica.com/documentation/index.html#!configuration/config-features <http://www.saxonica.com/documentation/index.html#!configuration/config-features>

The default limit is 10,000 and as mentioned in the documentation "when this limit is exceeded, no more bytecode is generated, and queries/stylesheets are interpreted instead."

This limit can be changed by using the setConfigurationProperty method on the Configuration class.

Having the assertion in we may have to review.

kind regards,

O’Neil

Post by Rademacher, Gunther
In my testing environment, Java assertions are enabled by default. This caused Saxon-EE to throw an AssertionError, when the limit imposed by FeatureKeys.MAX_COMPILED_CLASSES is exceeded.
There is no such error in the production (server) environment, but I wonder how to set MAX_COMPILED_CLASSES. It looks to me like this is a hard limit, that prevents further bytecode generation once it has been reached. As I have no indication of generated classes not being garbage collected, I tend to set it to Integer.MAX_VALUE. Would that return me to 9.7 behaviour, to this respect?
Also, is an assertion adequate for observing this limit? After all it addresses a situation that may legally occur, so refusing to continue seems a bit harsh to me, even when assertions have been enabled.
Best regards
Gunther
Software AG – Sitz/Registered office: Uhlandstraße 12, 64297 Darmstadt, Germany – Registergericht/Commercial register: Darmstadt HRB 1562 - Vorstand/Management Board: Karl-Heinz Streibich (Vorsitzender/Chairman), Eric Duffaut, Dr. Wolfram Jost, Arnd Zinnhardt, Dr. Stefan Sigg; - Aufsichtsratsvorsitzender/Chairman of the Supervisory Board: Dr. Andreas Bereczky - http://www.softwareag.com <http://www.softwareag.com/>------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org <http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________ <http://sdm.link/slashdot_______________________________________________>
saxon-help mailing list archived at http://saxon.markmail.org/ <http://saxon.markmail.org/>
https://lists.sourceforge.net/lists/listinfo/saxon-help <https://lists.sourceforge.net/lists/listinfo/saxon-help>

[O'Neil Delpratt]

Hi Gunther,

The assertion for observing the MAX_COMPILED_LIMIT is not necessary. We have added some logic to prevent its use so we now silently revert to the interpreted code if the limit is reached.

See bug issue and details of fix: https://saxonica.plan.io/issues/3349 <https://saxonica.plan.io/issues/3349>

kind regards,

O’Neil

Post by O'Neil Delpratt
Hi Gunther,
Details of the MAX_COMPILED_CLASSES is provided here: http://www.saxonica.com/documentation/index.html#!configuration/config-features <http://www.saxonica.com/documentation/index.html#!configuration/config-features>
The default limit is 10,000 and as mentioned in the documentation "when this limit is exceeded, no more bytecode is generated, and queries/stylesheets are interpreted instead."
This limit can be changed by using the setConfigurationProperty method on the Configuration class.
Having the assertion in we may have to review.
kind regards,
O’Neil

Post by Rademacher, Gunther
In my testing environment, Java assertions are enabled by default. This caused Saxon-EE to throw an AssertionError, when the limit imposed by FeatureKeys.MAX_COMPILED_CLASSES is exceeded.
There is no such error in the production (server) environment, but I wonder how to set MAX_COMPILED_CLASSES. It looks to me like this is a hard limit, that prevents further bytecode generation once it has been reached. As I have no indication of generated classes not being garbage collected, I tend to set it to Integer.MAX_VALUE. Would that return me to 9.7 behaviour, to this respect?
Also, is an assertion adequate for observing this limit? After all it addresses a situation that may legally occur, so refusing to continue seems a bit harsh to me, even when assertions have been enabled.
Best regards
Gunther
Software AG – Sitz/Registered office: Uhlandstraße 12, 64297 Darmstadt, Germany – Registergericht/Commercial register: Darmstadt HRB 1562 - Vorstand/Management Board: Karl-Heinz Streibich (Vorsitzender/Chairman), Eric Duffaut, Dr. Wolfram Jost, Arnd Zinnhardt, Dr. Stefan Sigg; - Aufsichtsratsvorsitzender/Chairman of the Supervisory Board: Dr. Andreas Bereczky - http://www.softwareag.com <http://www.softwareag.com/>------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org <http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________ <http://sdm.link/slashdot_______________________________________________>
saxon-help mailing list archived at http://saxon.markmail.org/ <http://saxon.markmail.org/>
https://lists.sourceforge.net/lists/listinfo/saxon-help <https://lists.sourceforge.net/lists/listinfo/saxon-help>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org <http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________ <http://sdm.link/slashdot_______________________________________________>
saxon-help mailing list archived at http://saxon.markmail.org/ <http://saxon.markmail.org/>
https://lists.sourceforge.net/lists/listinfo/saxon-help

[Rademacher, Gunther]

Hi O’Neil,

thank you.

One more question: would not “unlimited“ be a better default value for MAX_COMPILED_CLASSES?

It could then be set in environments where it is causing a problem, and ignored otherwise. 10,000 feels somewhat arbitrary. I did some testing, having set it to Integer.MAX_VALUE, and I did not see any problems with that.

Best regards
Gunther


From: O'Neil Delpratt [mailto:***@saxonica.com]
Sent: Donnerstag, 13. Juli 2017 14:23
To: Mailing list for the SAXON XSLT and XQuery processor
Subject: Re: [saxon] AssertionError when exceeding FeatureKeys.MAX_COMPILED_CLASSES in Saxon-EE 9.8

Hi Gunther,

The assertion for observing the MAX_COMPILED_LIMIT is not necessary. We have added some logic to prevent its use so we now silently revert to the interpreted code if the limit is reached.

See bug issue and details of fix: https://saxonica.plan.io/issues/3349

kind regards,

O’Neil

On 12 Jul 2017, at 16:32, O'Neil Delpratt <***@saxonica.com<mailto:***@saxonica.com>> wrote:

Hi Gunther,

Details of the MAX_COMPILED_CLASSES is provided here: http://www.saxonica.com/documentation/index.html#!configuration/config-features

The default limit is 10,000 and as mentioned in the documentation "when this limit is exceeded, no more bytecode is generated, and queries/stylesheets are interpreted instead."

This limit can be changed by using the setConfigurationProperty method on the Configuration class.

Having the assertion in we may have to review.

kind regards,

O’Neil

On 12 Jul 2017, at 12:48, Rademacher, Gunther <***@softwareag.com<mailto:***@softwareag.com>> wrote:

In my testing environment, Java assertions are enabled by default. This caused Saxon-EE to throw an AssertionError, when the limit imposed by FeatureKeys.MAX_COMPILED_CLASSES is exceeded.

There is no such error in the production (server) environment, but I wonder how to set MAX_COMPILED_CLASSES. It looks to me like this is a hard limit, that prevents further bytecode generation once it has been reached. As I have no indication of generated classes not being garbage collected, I tend to set it to Integer.MAX_VALUE. Would that return me to 9.7 behaviour, to this respect?

Also, is an assertion adequate for observing this limit? After all it addresses a situation that may legally occur, so refusing to continue seems a bit harsh to me, even when assertions have been enabled.

Best regards
Gunther


Software AG – Sitz/Registered office: Uhlandstraße 12, 64297 Darmstadt, Germany – Registergericht/Commercial register: Darmstadt HRB 1562 - Vorstand/Management Board: Karl-Heinz Streibich (Vorsitzender/Chairman), Eric Duffaut, Dr. Wolfram Jost, Arnd Zinnhardt, Dr. Stefan Sigg; - Aufsichtsratsvorsitzender/Chairman of the Supervisory Board: Dr. Andreas Bereczky - http://www.softwareag.com

[Michael Kay]

Post by Rademacher, Gunther
One more question: would not “unlimited“ be a better default value for MAX_COMPILED_CLASSES?
It could then be set in environments where it is causing a problem, and ignored otherwise. 10,000 feels somewhat arbitrary. I did some testing, having set it to Integer.MAX_VALUE, and I did not see any problems with that.

We introduced the limit because people with large stylesheets such as Dita and Docbook were experiencing memory problems due to the large number of classes compiled. It can also be a problem where a transformation service is executing an unlimited number of stylesheets during its life (I'm not sure what the current story is on garbage collection of loaded classes, but I think it's variable from one environment to another.)

JIT bytecode generation addresses part of the issue (large stylesheets with many unused template rules), but not the other part (unlimited number of stylesheets).

Michael Kay
Saxonica