Stop data retention! Click here & act! Are you a webmaster and want to participate? Here you can find all necessary material for your website - Willst du auch an der Aktion teilnehmen? Hier findest du alle relevanten Infos und Materialien:
Chris Karakas Online Forum Index Karakas Online
 FAQFAQ   Forum SearchForum Search   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
How to enter thousands of web links in PHP-Nuke


Goto page 1, 2, 3, 4  Next
 
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.
   Chris Karakas Online Forum Index -> PHP-Nuke Forum RSS Feed of this Forum
Share this page: These icons link to social bookmarking sites where readers can share and discover new web pages.Digg  del.icio.us  tc.eserver.org  Blinklist  Furl  Reddit  Blogmarks  Magnolia  Sphere  Yahoo!  Google  Windows Live  Technorati  Blue Dot  Simpy  Newsvine  Stumble Upon  co.mments.com  Blinkbits  BlogMemes  Connotea  View previous topic :: View next topic  
Author Message
chris
Dark Lord of the Sith


Joined: 10 May 2003
Posts: 6267
Location: Outer Space

PostPosted: Mon Aug 04, 2003 8:01 pm    Post subject: How to enter thousands of web links in PHP-Nuke
Reply with quote

Question Problem: You have gathered, or you have found Wink , thousands of web links that you would like to enter in the PHP-Nuke Web Links module. You don't want to do this manually.

Idea Idea: Use sed to produce the MySQL commands in a script file that you will then feed to MySQL.

Arrow Solution: Find the page that contains all the links you need. This may be a HTML file, but it may also be a CVS file from an exported MS Access data - this doesn't matter, the procedure is the same:

Edit the source with sed to convert every line that has a link and a description to an SQL command that inserts the link and its description to the aproppriate table. Very Happy

Example: Let's take the page
http://dmoz.org/Computers/Programming/Languages/PHP/Scripts/Content_Management/PHP-Nuke/

for an example. Save it as dmoz.html. Save the following sed script under the name "sedscr_downloads":

http://www.karakas-online.de/various/sedscr_downloads

Adapt sedscr_downloads to your situation (e.g. change cid from 100 to the right category id, add values for name and email, or change the date). Then, run it as follows:

Code:

sed -n -f sedscr_downloads dmoz.html > dmoz.sql


This will produce the following dmoz.sql:

http://www.karakas-online.de/various/dmoz.sql

As you can see, dmoz.sql contains the complete INSERT commands. Run it with

mysql < dmoz.sql

and all links will be inserted.
_________________
Regards

Chris Karakas
www.karakas-online.de
Back to top
View user's profile Send private message Send e-mail Visit poster's website
chris
Dark Lord of the Sith


Joined: 10 May 2003
Posts: 6267
Location: Outer Space

PostPosted: Tue Aug 05, 2003 11:14 am    Post subject:
Reply with quote

Since I used DMOZ (or ODP, Open Directory Project) as an example, let me add that there are a lot of scripts out there that can help you include the DMOZ data in your website.

An example is phpODP. phpODP enables you to add the content of The Open Directory Project at dmoz.org to your own website. You can let your visitors browse the categories, or search the directory - 100% locally on your site. The content is updated in real-time - so when ODP updates, you update!

There is even an ODP category that is devoted to scripts and tools that can scrap ODP data for you: Computers: Internet: Searching: Directories: Open Directory Project: Use of ODP Data: Upload Tools . It includes links to more than 20 tools that can do this, so I think one of those should fit your purposes, if the method described in my posting above is not the right one for you. In any case, it is a source of a wealth of ideas that you can explore!
_________________
Regards

Chris Karakas
www.karakas-online.de
Back to top
View user's profile Send private message Send e-mail Visit poster's website
chris
Dark Lord of the Sith


Joined: 10 May 2003
Posts: 6267
Location: Outer Space

PostPosted: Sat Aug 09, 2003 2:46 am    Post subject:
Reply with quote

You can now search the largest human-edited directory, the Open Directory Project, without leaving my site! The results are exactly the ones from the ODP - whenever they update, I update too!

You can reach it from "My online services" menu item (under "Links") of the Treemenu to the left, or from the "Services" link in the start page (if you are still there Wink ), or from the "Search My Homepage" link (you will have to scroll to the end there), or, directly by clicking on

http://www.karakas-online.de/phpodp/searchodp.php .

Try it!
_________________
Regards

Chris Karakas
www.karakas-online.de
Back to top
View user's profile Send private message Send e-mail Visit poster's website
chris
Dark Lord of the Sith


Joined: 10 May 2003
Posts: 6267
Location: Outer Space

PostPosted: Tue Nov 11, 2003 10:19 am    Post subject:
Reply with quote

I was searching the Web today, to see if somebody has made a PHP-Nuke module for ODP data and I found these two:

http://www.complex-berlin.de/download.php?op=getit&lid=846

http://www.complex-berlin.de/download.php?op=getit&lid=391

Both were found in:

http://www.complex-berlin.de/download.php?op=viewdownload&cid=43
_________________
Regards

Chris Karakas
www.karakas-online.de
Back to top
View user's profile Send private message Send e-mail Visit poster's website
peterc
Staff Sergeant
Staff Sergeant


Joined: 24 Feb 2004
Posts: 38

PostPosted: Sun Apr 11, 2004 2:52 pm    Post subject: sed
Reply with quote

Hi Chrish

Can you explain in "dummies" guide how this works.

I have obtained a list of links, saved it as html and saved the said script. How do I run sed? Is it thru DOS or XP.
Back to top
View user's profile Send private message
chris
Dark Lord of the Sith


Joined: 10 May 2003
Posts: 6267
Location: Outer Space

PostPosted: Sun Apr 11, 2004 10:03 pm    Post subject: Re: sed
Reply with quote

peterc wrote:
Open quoteHow do I run sed? Is it thru DOS or XP. Close quote


You run sed in Linux. Twisted Evil
_________________
Regards

Chris Karakas
www.karakas-online.de
Back to top
View user's profile Send private message Send e-mail Visit poster's website
sedscript
Lance Corporal
Lance Corporal


Joined: 11 Feb 2005
Posts: 6
Location: SD

PostPosted: Fri Feb 11, 2005 9:44 pm    Post subject: Help with Sedscript
Reply with quote

Im using a Virtual Unix Machine and im able to run the sed script successfully without any errors but the file dmoz.sql is not correct. It is a complete copy of dmoz.html
I also have to remove the -n in the sed command in order for it to work.
If I type:

Code:

sed -n -f sedscr_downloads.txt dmoz.html > dmoz.sql


then the dmoz.sql file is blank (0 bytes)
Im not sure if im doing this correctly. Does anyone have any ideas?

Thanks

[edit] Included code in code tags. Chris [/edit]
Back to top
View user's profile Send private message
chris
Dark Lord of the Sith


Joined: 10 May 2003
Posts: 6267
Location: Outer Space

PostPosted: Sat Feb 12, 2005 12:51 am    Post subject: Re: Help with Sedscript
Reply with quote

sedscript wrote:
Open quotebut the file dmoz.sql is not correct. It is a complete copy of dmoz.html
I also have to remove the -n in the sed command in order for it to work. Close quote


You should NOT remove the -n option! Removing it makes sed to print every line it processes - and since it processes the whole file without changing anything, the resulting file is an exact copy of the input file, i.e. of dmoz.html.

And why does sed not change anything? Well, the only explanation is that the sed script file sedscr_downloads.txt does NOT contain the right regular expressions to match the right patterns in the input file dmoz.html.

Maybe the format of dmoz.html has changed. Where did you get it from?
_________________
Regards

Chris Karakas
www.karakas-online.de
Back to top
View user's profile Send private message Send e-mail Visit poster's website
sedscript
Lance Corporal
Lance Corporal


Joined: 11 Feb 2005
Posts: 6
Location: SD

PostPosted: Mon Feb 14, 2005 3:44 pm    Post subject: sedscript
Reply with quote

I got the dmoz.html file from: http://dmoz.org/Computers/Programming/Languages/PHP/Scripts/Content_Management/PHP-Nuke/
and I saved it as dmoz.html
I got the sed script from: http://www.karakas-online.de/various/sedscr_downloads
and saved it as sedscr_downloads.txt
I uploaded the files to my ftp server
logged into my virtual unix program.
Logged into my ftp server.
downloaded the files
typed in:

Code:

sed -n -f sedscr_downloads dmoz.html > dmoz.sql


it created a dmoz.sql file and when I viewed the file it was blank.
0 bytes.

What do you think is going on?
Thanks

[edit] Included code in code tags. Chris [/edit]
Back to top
View user's profile Send private message
chris
Dark Lord of the Sith


Joined: 10 May 2003
Posts: 6267
Location: Outer Space

PostPosted: Thu Feb 17, 2005 4:06 pm    Post subject:
Reply with quote

Well, I've tried it again:

I got the list of PHP-Nuke-related sites from Open Directory:

http://dmoz.org/Computers/Programming/Languages/PHP/Scripts/Content_Management/PHP-Nuke/

saved it as

http://www.karakas-online.de/various/dmoz.html

Then I used the sedscr_downloads file:

http://www.karakas-online.de/various/sedscr_downloads

on a command-line together with sed:

I changed to the directory that contained both dmoz.html and sedscr_downloads and typed:

Code:

sed -n -f sedscr_downloads dmoz.html > dmoz.sql


This created the dmoz.sql file:

http://www.karakas-online.de/various/dmoz.sql

which is exactly what you need to insert all those PHP-Nuke-related sites into the $prefix_links_links table (I have just replaced my prefix with "prefix" in the tables' names, you should change "prefix" to your own prefix, as set in your config.php, in sedscr_downloads). You may need to take away some HTML tags (like <b> and </b>) from the titles in dmoz.sql, but that's all, you can then use it to populate the database, exactly as you did with nuke.sql when you installed PHP-Nuke (see Database creation).

You must be doing something wrong. Please try again and follow the exact steps as above.
_________________
Regards

Chris Karakas
www.karakas-online.de
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Share this page: These icons link to social bookmarking sites where readers can share and discover new web pages.Digg  del.icio.us  tc.eserver.org  Blinklist  Furl  Reddit  Blogmarks  Magnolia  Sphere  Yahoo!  Google  Windows Live  Technorati  Blue Dot  Simpy  Newsvine  Stumble Upon  co.mments.com  Blinkbits  BlogMemes  Connotea 
Display posts from previous:   
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.
   Chris Karakas Online Forum Index -> PHP-Nuke Forum
Page 1 of 4
This page contains valid HTML 4.01 Transitional - click here to check it!
This page contains a valid CSS - click here to check it!

 

Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group