Issue with Trac migration: "TracMigrate is not missing constant TracTicketChange!" (Bug #516)

I've been trying to update from Trac as well.

Specifically, Trac 0.12dev (which had been previously upgraded from Trac 0.11). In addition to the TracMigrate:: additions in the attached patch, I also had to change all the Time.at(some_datetime) calls to Time.at(some_datetime/1000000.0) - see http://trac.edgewall.org/ticket/9377 to handle differences in the way timestamps were stored prior to 0.12.

Then I ended up with the same :created_on error mentioned above. To get past that one, I had to change :created_on on line 489 to :created_at when creating a new Journal.

Now I'm struggling with JournalDetail being replaced by acts_as_journalized (see https://www.chiliproject.org/projects/chiliproject/wiki/Acts_as_journalized). Apparently this change in behavior in ChiliProject 2.0 and beyond breaks this script and quite frankly I am lost with regard to what needs to be done to fix this code.

Hello Mike,

thanks, with your patch apply I was able to migrate one of my old trac v 0.10.4-2 to chiliproject 2.4 but when I tried to migrate one that have ticket in then it does not work.

I had been able to migrate some other old trac v0.10 to chiliprojetct 1.2 but this seem to be a difficult job !

As some other have pointed out maybe it,s easier to migrate from trac to chiliproject 1.5.

Also, in your migration have you seen that the tables of content don't get migrate automaticly, and that tables have double | for each row separator ?

Thanks

I did notice that the table content and row separators didn't get migrated correctly as well as a lot of other wiki content but haven't had the time to look into it. In addition, I don't believe that the comments on the tickets are coming across correctly either.

I apologize for the incompletness of the patch but I was mainly concerned about getting open tickets migrated so that my group could continue development on our open projects. I have since taken to manually editing my wiki pages to correct them (I didn't have many) as I am not framilier with many of the differences between the wiki formatting in Trac vs Chiliproject. I'm still learning much about chiliproject as I was a long time Trac user. Once I get up to speed on some of those things and find a little bit of time, I'd be happy to look into all this a bit further. It would be nice to have a robust process to migrate from Trac to ChilProject.

Hi guys,

I had planned to watch/respond to this issue when it first popped up but it seems I missed it later on, sorry about that.

I haven't looked hard at the patches or anything yet, but in order to be able to integrate them into core I'd need some sample data I can use to test the migration, what would be even better is if someone was somewhat versed in what changes went into Trac (specifically the DB format) since the version the importer is compatible with.

If anyone of you could provide me with this data, I'd try to fix the importer between Christmas and new year.

Hi,

i got this error because I deleted some of my Trac users and the migration script seached for them while building the wiki and ticket journal.

We found a solution for that inside a Forum / Discussion thread:
https://www.chiliproject.org/boards/1/topics/556

ciao,
Chris

Hello,

We found a solution for that inside a Forum / Discussion thread:
https://www.chiliproject.org/boards/1/topics/556

installing an older ChiliProject version is not an option for me.

I am not sure if the migration script provided in #484 gets any steps further.
It already provides some trac 0.12 adjustments.

...but only the old ChiliProject (< 2.0) can currently import Wikis etc. correctly.

But you're right that importing Trac 12.x data only works with the patch for the new time format...

Relying on the patch for CP 2.x could be more time intensive and may require much more coding+debugging.

So: If you need a fast way, patching the CP 1.5 for the new time format will enable you to run the migration with acceptable results (as I and others already did).

Would that be a way for you?

ciao,
Chris

Hi chris,

I hava q question on this.

We have already ~ 20 projets in our productions ChiliProject v2.5 install. We can create an V1.5 CP install to migrate our trac info into the this transtion CP.

But then How to we import those Project into our production CP ?

If we could dump a project from ChiliProject V 1.5 into a tgz file and then import it into our production install it would be great...

Thanks for any input on this.

Guillaume

So: If you need a fast way, patching the CP 1.5 for the new time format will enable you to run the migration with acceptable results (as I and others already did).

Would that be a way for you?

ciao,
Chris

...just follow these steps:

- install CP 1.5.x and run the Trac migration on it (patch for new time format of Trac 12.x needed)
- update CP to 2.x and follow the upgrade documentation - which includes to run "rake db:migrate..."
...during the upgrade to a CP 2.x version all your data are converted into the new CP database format
- now you have a new CP 2.x system

There are only these problems:
- your Trac MUST be original with all users, wikis etc. and without someone doing write requests on it during migration
> a deleted Trac user would stop the migration with "nil pointer " errors
CP 1.5 currently can't handle the Trac 12.x issue timestamps, so CP 1.5 must be patched for that
- wiki syntax - especially tables - could be broken or a little bit distorted, which can be fixed later manually (linebreaks etc.)

So if you need help applying the patch for Trac 12.x timestamps, we need someone who could write that... otherwise (with Trac 11.x) you can start your migration immediately.

If you have further questions, just ask :) ...or look into the migration discussion linked in my former reply.

ciao,
Chris

This is all fine,

but how do you move or copy your trac project that is in you transition Chiliproct database to your production ChiliProject whe you already have ~ 20 projetc running.

Thanks,

Chris Dähn wrote:

...just follow these steps:

- install CP 1.5.x and run the Trac migration on it (patch for new time format of Trac 12.x needed)
- update CP to 2.x and follow the upgrade documentation - which includes to run "rake db:migrate..."
...during the upgrade to a CP 2.x version all your data are converted into the new CP database format
- now you have a new CP 2.x system

Hi,

the biggest problem is that dumping and importing a project isn't simple.

All projects have an incrementing numerical id -> see table "chiliproject.projects" (where "chiliproject" is your CP database).
This id is used in many other tables as "project_id", like in "wikis" and "issues".

So before you can migrate you must ensure that your newly migrated project gets a project_id, which isn't already in use inside your production environment.

Therefore just try to add as much empty projects as needed to ensure that the hightest project_id would be equal or higher, than the highest project_id in your production environment - e.g. with 20 projects your hightest id should be "20".

Check this by:
SELCT ID FROM chiliproject.projects;

Now grab a separate (testing) machine and start migrating based on a CP 1.5 and updating it to CP 2.x.

Then do full backup of target production environment (copy MySQL and CiliProject dirs).

Now add an empty Project with identical name of your migrated one - and check it's project_id which should be equal to the migrated one, e.g. here it could be "21" for your 21. project.

Shut down your production ChiliProject/Apache + Database/MySQL.

And now - that's much work - grab all entries for your migrated project, e.g. project_id 21.

I don't know all tables having a relation to the table "projects" (tables having the field "project_id"),
but you can easily determine that with the EXPLAIN sql command.

Then dump all tables with for "project_id=21" and insert them back into your production sql server.

That's so far as my simple understanding as user goes... maybe the developers have better/more infos or can correct me...

Hope that helps a little bit...

ciao,
Chris

Just brillant the idea to create empty project in the transition ChiliProject.

I was thinking of updating the project_id=21 manually but I can avoid that step using your trick.

Thanks again.

I was tinking of using the archiving utility in the project page but I have no idea how to import them back in a new server ;-) That woulb be a lot easier to move project from 1 server to another.

Bye

One thing I have to admit, after checking all my debug entries, is that I had to edit the app/models/journal.rb as well.

I do not really know why and what the consequences are but doing the following before "touching" check did the trick:

def touch_journaled_after_creation
+ journaled.touch unless journaled == nil ## HACK !!!
- journaled.touch
end


Is this going to make 3.1?

Working on this now. Is it reasonable to assume that anyone that has trac will be on 0.12 by now?

Felix Schäfer wrote:

Working on this now. Is it reasonable to assume that anyone that has trac will be on 0.12 by now?

I would say yes, because the migration for versions below Trac 0.12 is quiet easy.
As far as I experienced with our former Trac projects.

Hi,

I'm not sure, I have 4-5 trac running trac v0.11 and I can't migrate them now that I'm with chili ver 1.5 or higher. That used to be fine with older version of Chiliproject. I did not try you latest patch yet.

Has anobody been able to migrate trac into an old chiliproject installation and copy this chiliprojet projet to a newer chliproject where you already have other projetct running ( Chris Dähn has given some way to do it in note 14 #516note-14 ) ?

Thanks
Matthias Scholz wrote:

I would say yes, because the migration for versions below Trac 0.12 is quiet easy.
As far as I experienced with our former Trac projects.

Felix Schäfer wrote:

Working on this now. Is it reasonable to assume that anyone that has trac will be on 0.12 by now?

Bonjour Felix,

Any news on this ?

should I migrate my chiliproject install to 3.0.0 to test this out ?

note: an export / import project utility would be fanstastic to help us move chiliproject from server a to server b, anything plan

Thanks
Thanks

Felix Schäfer wrote:

Working on this now. Is it reasonable to assume that anyone that has trac will be on 0.12 by now?

Would a test trac help you working on this ? I could send you one of our test trac but I would prefer to do that in private.

Let me know, Guillaume

Working on this now. Is it reasonable to assume that anyone that has trac will be on 0.12 by now?

yes. Please make your migration code to work with 0.12 Trac. If user is on previous version he can download Trac and run trac update to 0.12.

I am also highly interested in Trac migration.

Guillaume Bourque wrote:

Felix Schäfer wrote:

Working on this now. Is it reasonable to assume that anyone that has trac will be on 0.12 by now?

Any news on this ?

Sorry for the slow updates, it's taking more time than I thought it was, but it's definitely high on my list. I'll try to at least throw in what I have already on github (this IS a work in progress though, so probably non-working often-rebased-and-force-pushed might-claim-the-soul-of-your-first-born type of thing…) if you're interested in trying it out.

should I migrate my chiliproject install to 3.0.0 to test this out ?

3.0.0 is released, so it will never be in there, hopefully in 3.1.0 though :-) (What I'd really really like is to make this an independent gem/executable though, but that's for later I guess…)

note: an export / import project utility would be fanstastic to help us move chiliproject from server a to server b, anything plan

That's not the scope of this issue, but if by "is there anything planned?" you meant "did anyone ever think this is a cool feature that could be added someday?": yes, if you meant "will this feature materialize in the near future?": it's not on our immediate roadmap and I can't even get my mind around the complexity of exporting plugins correctly and stuff that I'm not even sure it's gonna be doable in a "sufficient" scope.

Felix Schäfer wrote:

That's not the scope of this issue, but if by "is there anything planned?" you meant "did anyone ever think this is a cool feature that could be added someday?": yes, if you meant "will this feature materialize in the near future?": it's not on our immediate roadmap and I can't even get my mind around the complexity of exporting plugins correctly and stuff that I'm not even sure it's gonna be doable in a "sufficient" scope.

Hi Felix,

just having and export all issue and wiki and document with no pluging that could be imported in another chiliproject with the same version would be a very good start. We are free to reinstall all the plugins and adjust on the new server afterward !

Yes, I'm willing to test the trac patch on my current git clone if you tell me where to find the patch and how to apply it, I will do this in 1 hour.

Thanks,

Bye

Guillaume Bourque wrote:

Yes, I'm willing to test the trac patch on my current git clone if you tell me where to find the patch and how to apply it, I will do this in 1 hour.

I won't have the time to upload it until then, sorry, I'll make sure to update this issue when I do!

Holger Just wrote:

Pulling out of 3.1.0

Can you give information as to why ? This is a great feature having the ability ti migrate from trac !

Thanks

Guillaume Bourque wrote:

Can you give information as to why ? This is a great feature having the ability ti migrate from trac !

Simply because it wasn't ready on release day. This features continues to get developed. Once it's ready, it gets merged. You can help Felix by fixing the remaining issues on his branch.

Holger Just wrote:

Guillaume Bourque wrote:

Can you give information as to why ? This is a great feature having the ability ti migrate from trac !

Simply because it wasn't ready on release day. This features continues to get developed. Once it's ready, it gets merged. You can help Felix by fixing the remaining issues on his branch.

Ha thanks.

I would love to help Felix but I'm no developper at all, just an open source fan who knows linux and load balancer among other things !

Keep up the good work !

I tried your branch and got this error

• Invoke redmine:migrate_from_trac (first_time)
• Invoke environment (first_time)
• Execute environment
• Execute redmine:migrate_from_trac

WARNING: a new project will be added to Redmine during this process.
Are you sure you want to continue ? [y/N] y

Trac directory []: /home/hsn/trac/filezcom
Trac database adapter (sqlite, sqlite3, mysql, postgresql) [sqlite]: postgresql
Trac database host [localhost]: localhost
Trac database port [5432]:
Trac database name []: trac
Trac database schema [public]: filezcom
Trac database username []: trac
Trac database password []: melemekavu
Trac database encoding [UTF-8]:
Target project identifier []: FilezCOM
Unable to create a project with identifier 'FilezCOM'!
rake aborted!
undefined method `reload' for nil:NilClass
/home/hsn/chiliproject/lib/tasks/migrate_from_trac.rake:697:in `target_project_identifier'
/home/hsn/chiliproject/lib/tasks/migrate_from_trac.rake:771
/home/hsn/chiliproject/lib/tasks/migrate_from_trac.rake:754:in `prompt'
/home/hsn/chiliproject/lib/tasks/migrate_from_trac.rake:771
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/lib/rake/task.rb:205:in `call'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/lib/rake/task.rb:205:in `execute'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/lib/rake/task.rb:200:in `each'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/lib/rake/task.rb:200:in `execute'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/lib/rake/task.rb:158:in `invoke_with_call_chain'
/usr/local/lib/ruby/1.8/monitor.rb:242:in `synchronize'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/lib/rake/task.rb:151:in `invoke_with_call_chain'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/lib/rake/task.rb:144:in `invoke'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/lib/rake/application.rb:116:in `invoke_task'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `top_level'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `each'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `top_level'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/lib/rake/application.rb:133:in `standard_exception_handling'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/lib/rake/application.rb:88:in `top_level'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/lib/rake/application.rb:66:in `run'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/lib/rake/application.rb:133:in `standard_exception_handling'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/lib/rake/application.rb:63:in `run'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.9.2.2/bin/rake:33
/usr/local/lib/ruby/gems/1.8/bin/rake:19:in `load'
/usr/local/lib/ruby/gems/1.8/bin/rake:19
Tasks: TOP => redmine:migrate_from_trac

Radim Kolář wrote:

I tried your branch and got this error

Radim, thanks for testing, the error you get is actually here:

Target project identifier []: FilezCOM
Unable to create a project with identifier 'FilezCOM'!

The identifier must be lowercase (this should get tested/normalized but is not the current scope of this ticket).

Thanks for trying out though!

Has anybody had succes with the latest patch ?

would it be done via redmine ?

1) import trac to redmine

2) then import redmine to chiliproject 2.x or 3.x ?

Thanks

Guillaume Bourque wrote:

Has anybody had succes with the latest patch ?

would it be done via redmine ?

1) import trac to redmine

2) then import redmine to chiliproject 2.x or 3.x ?

As fare as I know and how I used the migration script, you do not need to convert you trac to redmine first. It should work directly.
I used trac 0.12 and converted to chiliproject 3.x, see post #16

Thanks

I had success with latest patch from this ticket. There are few minor things in markup which are not migrated correctly, but it worked for all my projects. I will open another ticket with converted wiki markup errors, but i think its in good shape to be committed into project.