Upgrade from 3.3RC2 to sprint 125

jeppebundsgaard

I have tried in multiple ways to upgrade. This latest try, I did the following:
1. unzipped TAO3.3RC2 to a new directory. Renamed it to tao33 as my previous installation
2. replaced composer.json with https://github.com/oat-sa/tao-community/blob/v0.125.3-alpha/composer.json
3. composer update'd (I did NOT install before the update - when I did that, I ended with compability problems)
4. Installed using the GUI. Using EXACT the same information as the previous installation (same instance name, same database and user).
5. I could now log in. But... I wanted to use my "old" data.
6. I replaced the new data folder with the previous one (I also tried to replace the config, but that gave errors - and I guess that when the usernames, instance name etc. are the same, it is better to keep the new version).
7. I copied every row from the previous statements, result_storage and variables_storage tables (I also tried to just replace the whole database) - in the new version there is one row in the kv_store - I have kept that.
8. I can go to the login in screen. I can even log in as a guest and see the public delivery I had in my previous installation. BUT...
9. When I log in as my own (admin) user, I get a 500 error. I cannot see anything in the logs.
10. I have tried to use both the previous and the new value as GENERIS_SESSION_NAME in generis.conf.php.

Please give me a hint on what to do next...

Cyril_Hazotte

We didn't provide yet a tutorial to upgrade from TAO 3.3 (sprint 96) to TAO 3.4 (sprint 116), so please wait a little bit until TAO 3.4 is released (no ETA yet).

jeppebundsgaard

The problem is that I need to get access to the bug fix of the duration problem (all durations above 1 minute are just reported as the seconds parts). And that bug fix depends on newer versions of generis and tao... So if you could help me to find out to just get access to that fix in 3.3RC2, it would be perfect.

Cyril_Hazotte

What is the exact TAO extension and eventually the commit number that brings this tweak?

jeppebundsgaard

I solved my problem by writing a script that downloads the student responses directly from the database - including the correct duration times.
So I don't need this update anymore, and I will just wait for the update and instructions.
Thanks,
Jeppe
PS. In the script I extract information about students, tests, deliveries, items and results into separate tables (more in line with a relational database approach than the RDF-approach in TAO), and pull the data from those. It made the export of data go very much faster. So if anyone gets frustrated by the download times from TAO, I can give you that script (you will have to adapt it, so it is not off-the-shelf-ready).

stephan_cgn

Hi jeppebundsgaard, I'm very interested in your script. Could you post it here?

jeppebundsgaard

Sorry stephan_cgn - I don't get emails from this forum, so I didn't see your request.

My solution was to first create new tables in the new database tao_results - here as export output from phpmyadmin:

 CREATE TABLE `delivery` (
   `id` int NOT NULL,
   `ressource` varchar(255) NOT NULL,
   `deliveryname` varchar(255) NOT NULL,
   `test` varchar(255) NOT NULL
 ) ENGINE=InnoDB DEFAULT CHARSET=latin1;

 CREATE TABLE `item` (
   `id` int NOT NULL,
   `ressource` varchar(255) NOT NULL,
   `itemname` varchar(255) NOT NULL
 ) ENGINE=InnoDB DEFAULT CHARSET=latin1;

 CREATE TABLE `result` (
   `id` int NOT NULL,
   `result_id` varchar(255) NOT NULL,
   `test_taker` varchar(255) NOT NULL,
   `delivery` varchar(255) NOT NULL,
   `test` varchar(255) NOT NULL,
   `item` varchar(255) NOT NULL,
   `identifier` varchar(255) NOT NULL,
   `value` longtext NOT NULL
 ) ENGINE=InnoDB DEFAULT CHARSET=latin1;

 CREATE TABLE `test` (
   `id` int NOT NULL,
   `ressource` varchar(255) NOT NULL,
   `testname` varchar(255) NOT NULL
 ) ENGINE=InnoDB DEFAULT CHARSET=latin1;

 CREATE TABLE `test_taker` (
   `id` int NOT NULL,
   `ressource` varchar(255) NOT NULL,
   `username` varchar(255) NOT NULL
 ) ENGINE=InnoDB DEFAULT CHARSET=latin1;

 ALTER TABLE `delivery`
   ADD PRIMARY KEY (`ressource`),
   ADD UNIQUE KEY `id` (`id`);

 ALTER TABLE `item`
   ADD PRIMARY KEY (`id`),
   ADD UNIQUE KEY `ressource` (`ressource`);

 ALTER TABLE `result`
   ADD PRIMARY KEY (`id`);

 ALTER TABLE `test`
   ADD PRIMARY KEY (`id`),
   ADD UNIQUE KEY `ressource` (`ressource`);

 ALTER TABLE `test_taker`
   ADD PRIMARY KEY (`ressource`),
   ADD UNIQUE KEY `id` (`id`);

 ALTER TABLE `delivery`
   MODIFY `id` int NOT NULL AUTO_INCREMENT;
 ALTER TABLE `item`
   MODIFY `id` int NOT NULL AUTO_INCREMENT;
 ALTER TABLE `test`
   MODIFY `id` int NOT NULL AUTO_INCREMENT;
 ALTER TABLE `test_taker`
   MODIFY `id` int NOT NULL AUTO_INCREMENT;

Then I extract information from tao33-database to tao-results:

--- Insert items into item table

insert into tao_results.item (ressource,itemname) (SELECT t.subject,t.object from statements t where t.subject in (SELECT s.subject  FROM `statements` s WHERE s.`predicate` LIKE 'http://www.tao.lu/Ontologies/TAOItem.rdf#ItemModel') and t.predicate LIKE "%#label%")

--- Insert tests into test table

insert into tao_results.test (ressource,testname) (SELECT t.subject,t.object from statements t where t.subject in (SELECT s.subject FROM `statements` s WHERE s.`predicate` LIKE 'http://www.tao.lu/Ontologies/TAOTest.rdf#TestTestModel') and t.predicate LIKE "%#label%") ON DUPLICATE KEY UPDATE testname=testname

--- Insert deliveries into delivery table

insert into tao_results.delivery (ressource,deliveryname) (SELECT t.subject,t.object from statements t where t.subject in (SELECT s.subject  FROM `statements` s WHERE s.`predicate` LIKE 'http://www.tao.lu/Ontologies/TAODelivery.rdf#DeliveryCompileTask') and t.predicate LIKE "%#label%") ON DUPLICATE KEY UPDATE deliveryname=deliveryname

--- Relate to test

update tao_results.delivery d left join tao33.statements s on s.subject=d.ressource set d.test=s.`object`  where predicate="http://www.tao.lu/Ontologies/TAODelivery.rdf#AssembledDeliveryOrigin"

--- Insert users into test_taker table

insert into tao_results.test_taker (ressource,username) (SELECT t.subject,t.object from statements t where t.subject in (SELECT s.subject  FROM `statements` s WHERE s.`object` LIKE 'http://www.tao.lu/Ontologies/TAO.rdf#DeliveryRole') and t.predicate LIKE "%#label%") ON DUPLICATE KEY UPDATE username=username

--- Insert data into result table

INSERT INTO `result` (`id`, `result_id`, `test_taker`, `delivery`, `test`, `item`, `identifier`, `value`) SELECT `variable_id`,`result_id`,`test_taker`,`delivery`,`test`,`item`,`identifier`,`value`
FROM tao33.variables_storage v
LEFT JOIN tao33.results_storage rs  ON (rs.result_id = v.results_result_id) where variable_id>$first_variable_id ON DUPLICATE KEY UPDATE id=id

And finally I can extract the results:

This is my query-string in PHP - I have a script which I use for a simple GUI to select delivery, test or user.

 $q="SELECT `result_id`,`username`, `testname`,`deliveryname`,`itemname`,`identifier`,FROM_BASE64(REGEXP_SUBSTR(REGEXP_SUBSTR(`value`, '(?:^.*\"(?:value|candidateResponse)\";s:[0-9]+:\")([^\"]*)'),'[^\"]*$')) as `val`
FROM result r
LEFT JOIN test_taker tt on r.test_taker=tt.ressource
LEFT JOIN delivery d on r.delivery=d.ressource
LEFT JOIN item i on r.item=i.ressource
LEFT JOIN test t on d.test=t.ressource
WHERE ".implode(" AND ",$wheres)." ORDER BY result_id";

Because of the regular expressions in this query, you need MYSQL 8.
And I also needed to set the mysql-variable regexp_time_limit=1024 in /etc/mysql/conf.d/mysql.cnf-file (or similar)