Old Board Import

Posted by dharlow 
dharlow – Old Board Import – December 12, 2007 12:14AM
I still would like to see if there is anyway to get the old boards imported into the new software. I might be able to commit some programmers to this assist in writing something to automate this if it would help.

Daniel



Edited 1 time(s). Last edit at 12/13/2007 11:03PM by tomierna.
tomierna (Admin)Re: Hard link to Original Spork Archive – December 12, 2007 04:44PM
dharlow:

I'm investigating ways of doing this now.

There are three approaches the way I see it:
1) Reverse-engineer the WebX export file and write scripts to parse it and insert the data into the Phorum tables.
2) Create WebX pages with WebX Template language in them to output delimited format for each forum/discussion, then write scripts to parse these files and insert the data into the Phorum tables.
3) Use WebX's NNTP output to input the messages into a database, and then use SQL to insert into the Phorum tables directly.

I think #3 would be easiest - probably using one of the existing PHP newsreaders, but instead of displaying, shoving the data into a MySQL table or two.

Toward this direction, I'm currently looking at http://mynewsgroups.sourceforge.net/ to see if it's db backend is sane enough to be a starting point.

If you have an NNTP reader (like Unison) you can point it at spork.macedition.com and use your spork username/password combo. You'll see the whole of Spork as a newsgroup.
dharlow – Re: Old Board Import – December 17, 2007 11:49AM
Tom,

What data would we need to import from the old boards to the new ones, obviously things like message body, date, user. Have you identified any other fields?

I have found some software that will pull a groups from a NNTP server into an SQL database but need to know more about what we need to import into phorum.

Daniel
tomierna (Admin)Re: Old Board Import – December 17, 2007 02:35PM
Daniel, to be honest, I haven't looked too deeply under the hood.

I can give you access to the database if you like - just PM me if that would be helpful.

Naturally, for testing purposes, I'd like to clone the db and forum so we can try the import as many times as we need to.

Phorum.com has a bunch of developer resources, and there have been some discussions about migrating from other board software that may be helpful.

The main thing that concerns me is that I believe that while posts are sorted by date, some of the code I've seen leads me to believe they use the sequential nature of message IDs to do some of the heavy lifting. We might have to do a "reverse import" where we start with a clean DB and load the old Spork data, and then load the new spork data on top of it...
tliet – Re: Old Board Import – December 18, 2007 12:47AM
Yesterday I've tried to login using an NNTP reader, but no dice...

What gives Tom, is NNTP turned off again in WebX?
tomierna (Admin)Re: Old Board Import – December 18, 2007 12:58AM
Ah. I neglected to open that port on the firewall. I sometimes forget to do that since my home IP has a hole punched through for more stuff than the general public.

It's open now. Try again?
dharlow – Re: Old Board Import – December 19, 2007 01:00AM
I got a connection error when tying to connect using Entourage. Just never responds.

Daniel
tomierna (Admin)Re: Old Board Import – December 19, 2007 12:09PM
Try now. I didn't realize port 119 UDP and TCP both had to be open.
dharlow – Re: Old Board Import – June 04, 2008 06:47AM
So I finally got around to working on this and have a pretty good idea on what needs to be done however when I went to download the archives from the old boards it will not connect to the news server. Did the port get closed or the server settings change?

Daniel
tomierna (Admin)Re: Old Board Import – June 04, 2008 10:20AM
Yeah, sorry Daniel. I think I shut the firewall rule because I thought it was stale.

Let me see about opening it up again.
tomierna (Admin)Re: Old Board Import – June 04, 2008 11:15AM
OK, it should be open now.
tliet – Re: Old Board Import – June 29, 2009 03:03AM
Hmm, are we getting spammers now?
Sorry, only registered users may post in this forum.

Click here to login