Database-Size


Omega
Posts: 48
Joined: Wed May 31, 2006 12:09 pm

Database-Size

Post by Omega » Thu Aug 24, 2006 12:53 pm

Hi !

Our Database is quite large (750MB) and I wonder, how do reduce file-size after cleaning the trash.
Any Hints ?

Thanks.


Yves
Posts: 17
Joined: Mon Aug 21, 2006 1:02 pm

Post by Yves » Thu Aug 24, 2006 5:20 pm

[quote=Thorsten]I believe the bottom line is, and has perhaps not been very well communicated, that online filing is an form filling and submitting tool but not an archive or document management tool and thus should not be used as such - especially as with growing data the application may become slower (I have no discrete performance data apart from user feedback). [/quote]
in http://forum.epoline.org/viewtopic.php?t=246
It seems you use eOLF to archive your filings. I think you must use another method to backup your data.
I'm studying a procedure where the user export the data after each filing and delete it from eOLF.
Experiences to share ?

Yves
Yves


Omega
Posts: 48
Joined: Wed May 31, 2006 12:09 pm

Post by Omega » Fri Aug 25, 2006 8:04 am

[quote="Yves"][
in [url]http://forum.epoline.org/viewtopic.php?t=246[/url]
It seems you use eOLF to archive your filings. I think you must use another method to backup your data.
I'm studying a procedure where the user export the data after each filing and delete it from eOLF.
[/quote]

We do not archive in epoline, but you can not force users to delete their filings. This means after we delete for example 60 Filings at once from the Database the database-file itself does not resize and there is no way in epoline to do that.

Of course we need the adressbook to be archived.
When we started exporting the entries, I was somehow annoyed that the different Plugins had different ways for exports.


Omega
Posts: 48
Joined: Wed May 31, 2006 12:09 pm

Post by Omega » Fri Aug 25, 2006 8:26 am

I need to correct myself.
It seems that we really do want to keep the filings in epoline in addition to our archives.


Oliver
Posts: 7
Joined: Fri Nov 23, 2007 12:57 pm
Location: London

Re: Database-Size

Post by Oliver » Mon Mar 01, 2010 12:31 am

Is there any way to automate the database compression/cleanout function? E.g. switches on the executeable which can be run via command line (scheduled task)? Having to do this manually is troublesome. We have a sproadic growth rate depending on the filings - it could be 100MB for several days and then bang... one day its 800MB all of a sudden.

Oli


Ckral
Posts: 21
Joined: Mon Jul 04, 2005 2:19 pm
Location: Munich
Contact:

Re: Database-Size

Post by Ckral » Mon Mar 01, 2010 11:54 am

I chime in because we are in the same boat. Our database reached epic proportions over the past years, somewhat around 10GB and was significantly slowing down the clients despite being hosted on a potent machine.
Manually exporting month per month was quite a chore after a while and I was looking for an automated solution. My early tests with a little selfwritten solution in pyton were successful however a solution to be released to the community would require the database structure to be fully documented and be available. My fear is that if a patch alters certain aspects of the tablespaces said solution would no longer work or not fully export everything properly.
A nice to have feature would imho would be to make the server manager accept command line params to allow batch exports for a defined date e.g.

olfsmanager /exp -t <credentials> -sl <storage location> -st <from ... to ...> -so <logfile>

or something like that.
Christoph Kral

IT-Department VJP Munich
Email: ckral@vjp.de


D. Van Haken
Posts: 70
Joined: Thu Jan 29, 2009 2:14 pm

Re: Database-Size

Post by D. Van Haken » Mon Mar 01, 2010 1:10 pm

One can automate the export and deletion (after export) of applications via the PMS interface.
For example, as called via the CMD interface:
To export an (= single) application identified by a unique application id:
OLFClient.exe -exportzip -Username=Administrator -Password= -Language=EN -ApplID=3

To delete a single application, identified by the application id (=3 in this case):
OLFClient.exe -remove -Username=Administrator -Password= -Language=EN -ApplID=3

The documentation on the PMS interface can be found in the downloadcentre on the epoline.org website, topic online filing: Online Filing V5 PMS development kit (for applicants and PMS providers).

One could, on a dailly basis query the system to find out which applications are to be archived (for example, those that are in the status SENT but where not archived yet) --> you will have a list of appid's to export.

Similar you could define the set of Appid's of applications which are already archived and could be removed (which have been there for more then x days/weeks). This will result in a list of applications (Appid's) to be deleted.

regards,
Dirk Van Haken

The EPO online filing client is not an archiving system, some house keeping must be done to avoid the size of the data-base to grow in an unbounded manner.


To query the internal db is not supported, any maintenance might break your system. The interface will protect you from this.
regards,

Dirk Van Haken
Product Manager Online Filing
EPO


Ckral
Posts: 21
Joined: Mon Jul 04, 2005 2:19 pm
Location: Munich
Contact:

Re: Database-Size

Post by Ckral » Mon Mar 01, 2010 1:44 pm

Dear Mr. Van Haken,

thank you for the information on this topic. Certainly we all are aware of the design philosophy of the EPOline filing application. However as I noticed over the past few years people tend to see client-server architectures with a a database on the backend of the concept as a medium to store and retrieve information for a longer period of time. I wonder how many patent firms and departments fill their firebird database to a degree where the application performance really takes a huge dive and leads to the enduser having an overall bad impression of the software itself.
Most of the people reading and posting here are IT professionals and will find a solution but peeking into the future there should be an easy to configure way to *unattended* export older applications for non-tech savvy user.
Christoph Kral

IT-Department VJP Munich
Email: ckral@vjp.de


alexthurgood
Posts: 58
Joined: Tue May 30, 2006 9:29 am

Re: Database-Size

Post by alexthurgood » Tue Sep 14, 2010 1:46 pm

Ckral wrote:I wonder how many patent firms and departments fill their firebird database to a degree where the application performance really takes a huge dive and leads to the enduser having an overall bad impression of the software itself.
Most likely all of those law firms and IP departments who do not have dedicated IT support staff specialised in DB maintenance, i.e. probably about 85% of them at a wild guess (almost certainly 100% for small IP law firms). Last time I looked, the EPO did not make it especially clear that eOLF was not a DMS, CMS or any kind of other document or content management system designed for long term and cumulative storage of data sets. This really ought to be made mention of more clearly, but of course, by doing so, any such warnings are likely to decrease uptake of eOLF.

Quite frankly, having tried to file applications using eOLF and having the app crash on me repeatedly after filling in virtually all of the forms, I found it quicker, safer, and far less frustrating to fill out a PDF form and file by fax. Ask me today where eOLF has saved me time and money and I would say, nowhere, but then again, perhaps my needs are not those of a large corporation (in fact almost certainly so). In this thread alone, I have learned that there is no auto-vacuuming on shutdown or even periodically for example, which would help improve database performance. Is there not a firebird server startup parameter of configuration entry that would deal with this though ?

Alex Thurgood


Locked