<?xml version="1.0" encoding="UTF-8"?><!-- generator="WordPress/2.9.2" -->
<rss version="0.92">
<channel>
	<title>G2 Crawler News</title>
	<link>http://crawler.trillinux.org/news</link>
	<description>G2 Crawler News</description>
	<lastBuildDate>Wed, 21 Oct 2009 21:01:16 +0000</lastBuildDate>
	<docs>http://backend.userland.com/rss092</docs>
	<language>en</language>
	
	<item>
		<title>A different way to look at the network</title>
		<description><![CDATA[Since the middle of August the crawler has been recording the time when hubs join and leave the network. This allows for certain time based trends to be realized. The hub is identified by its IP address. One way to visualize a set of IP addresses is with the Hilbert curve which was made popular [...]]]></description>
		<link>http://crawler.trillinux.org/news/2009/10/09/a-different-way-to-look-at-the-network/</link>
			</item>
	<item>
		<title>Two new experimental features</title>
		<description><![CDATA[A few weeks ago two new features were released. The first is a world map view of the country page and the second shows how the network size changes over time.
World Map
The world map shows two different data sets. Red circles represent where hubs say they are located. The size of the circle indicates how [...]]]></description>
		<link>http://crawler.trillinux.org/news/2009/10/03/two-new-experimental-features/</link>
			</item>
	<item>
		<title>Network size</title>
		<description><![CDATA[The network size is now featured on the front page once again. When the new crawler was implemented that statistic had to be dropped because it was too resource intensive to calculate with how the new crawler worked. But now that issue has been resolved.
Some background
The number of leaves on the network isn&#8217;t a good [...]]]></description>
		<link>http://crawler.trillinux.org/news/2009/05/25/network-size/</link>
			</item>
	<item>
		<title>A Quick Update</title>
		<description><![CDATA[I haven&#8217;t made a post in awhile so I thought I should.
Not much is going on with the crawler right now. I&#8217;ve been pretty busy lately and haven&#8217;t had any time to spend on improving the crawler. However there were a few subtle updates to many of the webpages. More detailed descriptions were added to [...]]]></description>
		<link>http://crawler.trillinux.org/news/2009/02/12/a-quick-update/</link>
			</item>
	<item>
		<title>The Architecture of a Crawler</title>
		<description><![CDATA[I&#8217;m going to explain how crawlers work. There are three main tasks that a crawler has to take care of.

Find new hosts to crawl.
Request data from a host that is being crawled.
Display to the user the data gathered.

This design lends itself well to being distributed. Several host crawlers (those that perform task 2) can all [...]]]></description>
		<link>http://crawler.trillinux.org/news/2008/11/01/the-architecture-of-a-crawler/</link>
			</item>
	<item>
		<title>Recent Updates</title>
		<description><![CDATA[My focus lately has been on hub uptimes. There is a new page showing hub uptime distribution graphs. It gives a visual representation of some of the categories on the uptimes page. The overall hub uptime distribution graph also features two vertical lines. The red line shows where the average hub uptime is and the [...]]]></description>
		<link>http://crawler.trillinux.org/news/2008/10/19/recent-updates/</link>
			</item>
	<item>
		<title>Quick g2paranha update</title>
		<description><![CDATA[The crawler has been running pretty well with only minor tweaks from day to day which sometimes show up as blips in the graph. It was also down for a few days due to a failing hard drive.
Yesterday the crawler got into the Foxy network again which uses the same protocol as G2 but is [...]]]></description>
		<link>http://crawler.trillinux.org/news/2008/07/10/quick-g2paranha-update/</link>
			</item>
	<item>
		<title>g2paranha &#8211; The New G2 Crawler</title>
		<description><![CDATA[Anyone who has read through this blog knows that the crawler has tended to crash fairly often. In recent times it was crashing to much to even continue running it. But rather than give up entirely I decided to write my own crawler. Five weeks later and g2paranha has emerged. To go along with the [...]]]></description>
		<link>http://crawler.trillinux.org/news/2008/06/23/g2paranha/</link>
			</item>
	<item>
		<title>The State of G2</title>
		<description><![CDATA[I was reading the Gnutella2 article on Wikipedia today and I noticed both entries in the External Links section point to my sites (crawler.trillinux.org and g2.trillinux.org). The latter being the new home for the G2 specs after gnutella2.com was allowed to expire. This got me thinking that it looks like I&#8217;m the only one trying [...]]]></description>
		<link>http://crawler.trillinux.org/news/2008/02/28/the-state-of-g2/</link>
			</item>
	<item>
		<title>More Crawler Downtime</title>
		<description><![CDATA[I spent last weekend replacing my router with another computer. The transition was a bit bumpy but things are starting to get sorted out. More extended periods of downtime are possible over the next few weeks as I get things completely transitioned and working reliably.
]]></description>
		<link>http://crawler.trillinux.org/news/2008/01/30/more-crawler-downtime/</link>
			</item>
</channel>
</rss>

<!-- Dynamic Page Served (once) in 1.431 seconds -->
