Forum

Slow delivery result export

edited June 21 in Support

Hello! We have issues with TAO being very slow and resource intensive when exporting to the point where a standard PHP configuration simply fails because of memory_limit and max_execution_time. it seems like both php and sql is fighting for full CPU usage for minutes when exporting this delivery (delivered to 1000 test takers).

Is it possible to export CSV data by local scripts instead? Any plans on a 1nf normalized database scheme or replacement of sql in favor of noSQL in the future?

Any support for memcache :P ?

Comments

  • As always we are open to contributions from the community but at this time there are no plans to replace the usage of a SQL based database or add memcache.

    As far as using a script, depending on the data you are trying to export, you should be able to write a script that does a dump to a CSV. You could also adjust your SQL and PHP settings but it depends on cores, ram, etc. how much you can adjust for.

  • I would love to contribute, but there is no proper developer documentation and the project is honestly a mess that I'd rather not touch without getting paid. You build your product around RDF instead of building RDF into your product. As a result your codebase is more or less impossible to read and comprehend as an outsider, everything is just an core_kernel_classes_Resource, your disgrace of a database schema is a few key value stores with useless indexes and terrible performance, which I suspect is the primary reason that exporting mere 1000 lines of CSV takes three minutes and about 500mb of ram...

    I honestly tried to contribute, but I'd rather not.

Sign In or Register to comment.

We use cookies on our website to support technical features that enhance your user experience. Cookie Policy Privacy Policy

×