irmis-devel Mailing List for IRMIS

Hi all,

 

In the spirit of release early release often, I am planning to release
once a month. Therefore I just packaged and released 3.0.1-alpha. From
the history page:

3.0.1-alpha (r520) 2008-11-26

*	Schema: merge cmpnt and cmpnt_installation.
*	Schema: added key/value pair properties to component and
component types
*	Service: now using Java Persistent Architecture (JPA) along with
straight JDBC
*	Service: implemented query that returns few components at a time
*	Java API: listeners for local cache refresh
*	Java API: server location can now be specified by environment
variables or JVM properties
*	Applets: using IRMIS logo as a splash screen
*	Applets: all widget now support delayed loading (removed the
"loading" screen with progress bar)
*	Applications: all updated to support component and component
type properties
*	Cable editor: first implementation

Gabriele

Schema change was implemented. If you use the new version, you'll have
to recreate the schema in mysql. We don't have an update script,
though...

 

Gabriele

 

From: Carcassi, Gabriele 
Sent: Wednesday, November 19, 2008 4:25 PM
To: irm...@li...
Subject: [Irmis-devel] Schema change

 

Hi all,

 

As you may or may not know, we are working on a schema change in the
component and component type area. The main changes are to merge
together cmpnt and cmpnt_installation, and to add support for key-value
pairs to component types.

 

Yesterday I got the schema from Don, and I am implementing the changes
to the code, which I'll probably commit tomorrow.

 

Gabriele

Hi all,

 

As you may or may not know, we are working on a schema change in the
component and component type area. The main changes are to merge
together cmpnt and cmpnt_installation, and to add support for key-value
pairs to component types.

 

Yesterday I got the schema from Don, and I am implementing the changes
to the code, which I'll probably commit tomorrow.

 

Gabriele

Hi all,

 

I finished my evaluation on how to best use Java Persistence
Architecture (JPA) within IRMIS. I reimplemented the streaming of all
the components and relationships, and the performance loss is around
6.13%. (Details are attached in the excel spreadsheet if anybody cares).

 

This was measured by running the query 5 times and taking the average.
To compare, I run against both the training dataset and the full dataset
from Argonne. The difference was nonexistent in the training set,
confirming that this is an added cost per result entry instead of per
query (the cost per query does not seem to be affected and is probably
far superior that this small increase). The difference between the
latency seen by the server and by the client was not affected, which
means the overall machinery is working exactly the same.

 

Memory effects on the server are a bit more complicated, but in the end
of the day are the same. Using JDBC, it seems I was careful enough that
some of the resources allocated at query time could be released right
away. So, if the server would start with 30/35MB, would peak at 90/110MB
and then go down right away at 80/85MB. With JPA, the release is not
there, and it goes from 30/35MB to 115/120MB right away. The extra
memory is most likely due to the extra mapping objects that were not
required in JDBC.

 

I think database independence justifies that cost. 

 

Unfortunately the code becomes a bit more verbose: one has to provide
the classes to map the query onto, and the xml mapping has to be
implemented for each entity since all attempts to generalize incurred in
too much performance penalties. Tried to use introspection for the xml
mapping, but it would decrease performance by 60%. Putting the data in
maps instead of fields took 20% and putting them in arrays took 15%. I
don't think that's worth the effort.

 

I'll keep looking into other options, but I am not holding my breath.

 

So, to recap, these are the restrictions on how we should use it:

1) No business logic in the JPA classes, so that they can be
re-generated from the database at any time

2) For reading, one has to still think very carefully at how the queries
are done so that the number is minimized (just using the "natural" way,
produced code that was an order of magnitude less efficient because it
trigger more then one query for each result).

3) Create classes specific for the query if that's needed

 

The trade-off in using JPA:

1) For writes, code becomes far more readable (and we are a lot less
concerned about performance there).

2) For reads, code becomes more verbose, the queries become a lot more
readable (avoid many joins), and performance is not affected that much

3) More database independence

 

Gabriele