search engine

A search engine is a web-based tool that enables users to locate information on

the World Wide Web. Popular examples of search engines are Google, Yahoo!, and
MSN Search. Search engines utilize automated software applications (referred to
as robots, bots, or spiders) that travel along the Web, following links from page to
page, site to site. The information gathered by the spiders is used to create a
searchable index of the Web.

Search engines are programs that search documents for specified keywords and
returns a list of the documents where the keywords were found. A search
engine is really a general class of programs, however, the term is often used to
specifically describe systems like Google, Bing and Yahoo! Search that enable
users to search for documents on the World Wide Web.
Web Search Engines
Typically, Web search engines work by sending out a spider to fetch as many documents as possible. Another
program, called an indexer, then reads these documents and creates an index based on the words contained in each
document. Each search engine uses a proprietary algorithm to create its indices such that, ideally, only meaningful
results are returned for each query.
As many website owners rely on search engines to send traffic to their website, and entire industry has grown around
the idea of optimizing Web content to improve your placement in search engine results. Learn more about search
engine optimization (SEO) in this Webopedia' definition.

Common Search Engine Types

In addition to Web search engines other common types of search engines include the following:

Local (or offline) Search Engine: Designed to be used for offline PC, CDROM or LAN searching
usage.

Metasearch Engine: A search engine that queries other search engines and then combines the
results that are received from all.
Blog Search Engine: A search engine for the blogosphere. Blog search engines only index and
provide search results from blogs (Web logs).

2. Different types of search engines

When people mention the term "search engine", it is often used generically to describe both crawler-based search engines and human-powered
directories. In fact, these two types of search engines gather their listings in radically different ways and therefore are inherently different.

Crawler-based search engines, such as Google, AllTheWeb and AltaVista, create their listings automatically by
using a piece of software to crawl or spider the web and then index what it finds to build the search base. Web

page changes can be dynamically caught by crawler-based search engines and will affect how these web pages
get listed in the search results.
Crawler-based search engines are good when you have a specific search topic in mind and can be very efficient
in finding relevant information in this situation. However, when the search topic is general, crawler-base search
engines may return hundreds of thousands of irrelevant responses to simple search requests, including lengthy
documents in which your keyword appears only once.
Human-powered directories, such as the Yahoo directory, Open Directory and LookSmart, depend on human
editors to create their listings. Typically, webmasters submit a short description to the directory for their websites,
or editors write one for the sites they review, and these manually edited descriptions will form the search base.
Therefore, changes made to individual web pages will have no effect on how these pages get listed in the search
results.
Human-powered directories are good when you are interested in a general topic of search. In this situation, a
directory can guide and help you narrow your search and get refined results. Therefore, search results found in a
human-powered directory are usually more relevant to the search topic and more accurate. However, this is not
an efficient way to find information when a specific search topic is in mind.
Table 1 summarizes the different types of the major search engines.

Search Engines

Types

Google

Crawler-based search engine

AllTheWeb

Crawler-based search engine

Teoma

Crawler-based search engine

Inktomi

Crawler-based search engine

AltaVista

Crawler-based search engine

LookSmart

Human-Powered Directory

Open Directory

Human-Powered Directory

Yahoo

Human-Powered Directory, also provide crawler-based search

results powered by Google

MSN Search

Human-Powered Directory powered byLookSmart,

also provide crawler-based search results
powered by Inktomi

AOL Search

Provide crawler-based search results powered by Google

AskJeeves

Provide crawler-based search results powered by Teoma

HotBot

Provide crawler-based search results powered

by AllTheWeb, Google, Inktomi and Teoma, 4-in-1 search
engine

Lycos

Provide crawler-based search results powered by AllTheWeb

Netscape Search

Provide crawler-based search results powered by Google

Table 1: Different types of the major search engines

From the table above we can see that some search engines like Yahoo and MSN Search provide both crawler-based results and human-powered
listings, therefore become hybrid search engines. A hybrid search engine will still favor one type of listings over another as its type of main results.
There is another type of search engines that is called meta-search engines.
Meta-search engines, such as Dogpile, Mamma, and Metacrawler, transmit user-supplied keywords simultaneously to several individual search
engines to actually carry out the search. Search results returned from all the search engines can be integrated, duplicates can be eliminated and
additional features such as clustering by subjects within the search results can be implemented by meta-search engines.

Meta-search engines are good for saving time by searching only in one place and sparing the need to use and learn several separate search
engines. "But since meta-search engines do not allow for input of many search variables, their best use is to find hits on obscure items or to see if
something can be found using the Internet."

3. Major components of crawler-based search engines

Searching with the aforementioned three different types of search engines are different processes therefore comparing search features and
search results between them will be inappropriate. I will focus on crawler-based search engines for my comparisons in this article. From this point
on, crawler-based search engines and search engines can be used interchangeably in this article without loss of clarity.
Crawler-based search engines have three major components.
1) The crawler
Also called the spider. The spider visits a web page, reads it, and then follows links to other pages within the site. The spider will return to
the site on a regular basis, such as every month or every fifteen days, to look for changes.
2) The index
Everything the spider finds goes into the second part of the search engine, the index. The index will contain a copy of every web page that
the spider finds. If a web page changes, then the index is updated with new information.
3) The search engine software
This is the software program that accepts the user-entered query, interprets it, and sifts through the millions of pages recorded in the index
to find matches and ranks them in order of what it believes is most relevant and presents them in a customizable manner to the user.
All crawler-based search engines have the basic parts described above, but there are differences in how these parts are tuned. That is why the
same search on different search engines often produces different results. Our comparisons will then be based on these differences in all three
parts.

Internet Relay Chat

INTERNET RELAY CHAT (IRC) IS AN APPLICATION LAYER PROTOCOL THAT FACILITATES
COMMUNICATION IN THE FORM OF TEXT. THE CHAT PROCESS WORKS ON A
CLIENT/SERVER NETWORKING MODEL. IRC CLIENTS ARE COMPUTER PROGRAMS THAT A
USER CAN INSTALL ON THEIR SYSTEM. THESE CLIENTS COMMUNICATE WITH CHAT
SERVERS TO TRANSFER MESSAGES TO OTHER CLIENTS.[1] IRC IS MAINLY DESIGNED
FOR GROUP COMMUNICATION IN DISCUSSION FORUMS, CALLED CHANNELS,[2] BUT ALSO
ALLOWS ONE-ON-ONE COMMUNICATION VIA PRIVATE MESSAGES[3] AS WELL AS CHAT AND
DATA TRANSFER,[4] INCLUDING FILE SHARING.[5]
CLIENT SOFTWARE IS AVAILABLE FOR EVERY MAJOR OPERATING SYSTEM THAT SUPPORTS
INTERNET ACCESS.[6] AS OF APRIL 2011, THE TOP 100 IRC NETWORKS SERVED MORE THAN
HALF A MILLION USERS AT A TIME,[7] WITH HUNDREDS OF THOUSANDS OF
CHANNELS[7] OPERATING ON A TOTAL OF ROUGHLY 1,500 SERVERS[7] OUT OF ROUGHLY 3,200
SERVERS WORLDWIDE.[8] IRC USAGE HAS BEEN DECLINING SINCE 2003, LOSING 60% OF ITS
USERS (FROM 1 MILLION TO ABOUT 400,000 IN 2014) AND HALF OF ITS CHANNELS (FROM
HALF A MILLION IN 2003).

What is Usenet?

Usenet is a collection of newsgroups where the users can

post messages and these posted messages are distributed
via Usenet servers. Each server holds these posted
messages for a certain period of time. This period of time
is also known as retention time. Usenet can be compared
with an internet forum, but the main difference is that the
messages are not stored in one server and are only
available for a certain amount of time.
A user posts a message in a certain channel (e.g.
newsgroup) and then this message will be
distributed via the different newsgroup servers.
The other users can login anytime to one of these
servers via special software (newreader) and see
all new messages in that channel. They can also
reply to the newly posted messages and these
new posts will also be distributed via Usenet
servers. If the retention time of a certain server
is 30 days, then these messages will be available
just for 30 days in that server. Start
downloading! In order to start downloading you
will need usenet access software. Please click on
the button on the right side to download free
usenet access software, which allows you to
download 1000 MB on full speed up to 100
Mbits/s!
Downloading from Usenet
Perhaps you have already heard about making a download from Usenet. Downloading works quite similar to
post messages. Only a so-called binary file (or short binary) will be posted. The file posting usually occurs in
binary newsgroups, which are called binaries. You can recognize these binary newsgroups from the newsgroup
name, for example, alt.binaries.games. In order to read messages, reply to messages and download posted files
you need a usenet reader. It is often complex software which needs to be assembled with the downloaded files
(that are often posted in chunks) later on. Due to the structure of usenet and the fact that downloading and
merging these files could only be done via difficult software, it was long just a playground for computer geeks.
Today however, many good alternatives are available, which can be operated easily with the Usenet software. A
good example for that is the UseNeXT software which automatically merges partial files. Also it makes use of
dedicated servers, so you can download as much as you like at your full connection speed for just a small
monthly fee. Retention time of UseNeXT servers is over 1000 days. Read more about how to download from
Usenet on usenet.org.

How does Usenet work?

Long before the World Wide Web was even a vague but exciting idea, Usenet served as the worlds biggest discussion board.
These days Usenet also contains the largest collection of user generated files on the globe. Therefore, Usenet is composed of
text based newsgroups with discussions about any topic you can imagine, and user supplied files like music, software, videos
and eBooks.
Inherently, Usenet is a decentralized network for the distribution of postings. Using it is very simple, as the user only needs to
search for the topic hes interested in. Hundreds of millions of messages or articles are categorized and distributed into socalled newsgroups. This content is stored on millions of Usenet server computers and forms the core of Usenet. Its method is
based upon each news server sharing messages and articles with other Usenet news servers, to ensure that every news
service provider possesses the latest and most current copy of a message. Therefore the servers often provide the same data
differing only retention time. That means, some servers can provide you with a message for up to three years while others
delete it automatically after a few days. And not only Usenet works like this, news servers are connected in a decentralized way
too and exchange information with each other. Indeed theres no duplicate content on the basis of the facts of distinct message
IDs.
For the record, there is no one in charge of the whole system. However theres always somebody in charge of a particular news
server. The appeal of Usenet is based on its decentralized structure as well as its storage time and fashion of spreading articles
and files. Just keep in mind that Usenet doesnt store content and postings but rather distributes them.

How to start with Usenet?

To use the services of Usenet, some conditions have to be fulfilled. Generally there are three things you will
need. Internet access is a must have, of course. There are no limits, but if you plan on using Usenet especially
for binary-newsgroups, the speed and quality of the connection your Internet service provider offers can affect
your user experience. Also a flat rate might certainly be convenient. That way you can use your bandwidth up
to the maximum.
Furthermore a news server is required. These differ in terms of costs and services. Internet access providers
often offer a free news server with their services, but those are almost exclusively available without binary
newsgroups. If you want to use binary newsgroups, its inevitable to subscribe to a professional Usenet provider
at your own expense. They will allow you to experience all of Usenets facets for only a small fee.
Finally a newsreader is essential. This software is needed to read messages or to download files from the
Usenet. Nowadays some mail clients, like Thunderbird or Outlook Express, offer you the possibility of reading
articles in text-based newsgroups. If that is all you want from Usenet, you probably wont need a special client.
If you are interested in downloading files, though, you will want to look for a newsreader that can handle binary
files, like HolmeZ. Often these newsreaders are offered by your provider anyways, but sometimes you are free
to choose according to your individual needs. If all those requirements are fulfilled, you can explore the
unlimited world of Usenet.

Is Usenet legal?
Usenet and the access to Usenet is completely legal. Usenet is comparable with the Internet and serves for
exchanging information in a legit way. Similar to the Internet, Usenet offers the opportunity for downloads. The
question about legality comes predominantly in combination with the usage of newsgroups from the subhierarchy alt.binaries, which was cut out to be specialized for spreading binary data attachments.
The usage of Usenet, consisting of subscribing to newsgroups, reading or commenting of postings plus
communication, in the case of no slanderous content or human rights violations, is entirely legal. Even
downloading files is in most cases permissible, as long as the movies, programs or files are unrestrictedly
available and dont cause any conflicts with copyright law. Consequently complications can merely occur in
binary-newsgroups, because all sorts of file attachments are allowed here. If the author of the posting isnt the
originator of the provided data, the author acted illegal. The poster violated the law of copyright and can be

liable to prosecution. But contrary to P2P-filesharing, using a premium-account of Usenet is entirely anonymous,
due to SSL encrypted connections and closed lines.
Eventually misconduct of a few individuals users leads to the connection of the terms illegality and Usenet. If
you want to use Usenet only with the intention of reading and commenting posts, or as a tool of
communication, you can do this without compunction. Not legal is solely the download of scattered files
protected by copyright law but Usenet by itself is completely legitimate and licit.

Whats the difference between

Usenet and BitTorrent?
When it comes to downloading files, you will hear the names of Usenet and BitTorrent again and again. But
whats the difference between these platforms? If you only consider the costs, BitTorrent might be your first
choice. However if you are researching the quality features both offer, the differences become clear really fast.
Usenet is definitely the better choice, due to quite a lot of benefits.
Looking at the bandwidth available you can realize a first significant difference at once. Usenets download
speed is only limited by the bandwidth of your Internet service provider and can lead to velocity advantages 10
times higher than usual downloads. Data download takes places directly from servers. When using BitTorrent
your download speed is depending on other users internet connection as well as the amount of seeds.
Furthermore Usenet can be used completely anonymous, since your IP address cant be accessed from outside.
Your Usenet provider usually doesnt keep any log files about your activities or download actions. They solely
gather data about your amount of traffic to render account for your package. Apart from reasons of data
protection, logging activities of their customers would simply take way too much storage space. If you regard
higher security and anonymity as benefits, Usenet is your way to go. Torrents make it comparatively easy to
find out who conducts which download due to the lack of SSL encryption. As you dont have to distribute any
data to the Usenet, like it is required in P2P networks like BitTorrent, there are no data about your whereabouts
available.
Furthermore Usenets longer retention time is an important argument. Retention time of articles and data
usually differ by provider but often adds up to several years, while storage time by torrents is down to only
approximately one month. As an extra treat, Usenet provides the opportunity of repairing faulty or uncompleted
files with par-data.
The most common drawback you will hear about is the fee you have to pay to your Usenet provider. If you want
a save BitTorrent access though, you will have to invest into a VPN. That in mind the costs accommodate.
Considering the advantages listed above, Usenet might be the network of your choice. Anyway Usenet cares
about higher velocity, your security, and your privacy.

Which are the leading Usenet

providers?
There are hundreds of Usenet providers, but which ones are the leading companies? Many providers offer
nearly the same and differ mainly in retention time, payment and support. As the markets leaders UseNeXT,
Usenet.nl and Giganews have to be named, you should not forget about Firstload, UsenetServer, Newshosting
and aEton.
There are, of course several other providers previously not mentioned. Newshosting, for example, offers only 60
connections at the same time in their XL package. Regarding common offers, UseNeXT, for example, supplies
30 parallel connections. The average number of connections comes to 15. In terms of bandwidth, SSL
encryption, amount of newsgroups and account sharing, all of the 7 providers named above are following nearly
identical ways. Shared accounts are prohibited everywhere. You can use your full bandwidth until you reach

your traffic limit. In standard bundles SSL encryption is generally included. Every provider usually offers you
about 100,000 newsgroups.
One of the more important criteria is the retention time of text and binary files. Giganews for example can offer
you a storage period of 8 years, followed by UseNeXT with 3,832 days retention time, while the average is
about 2,400 days. Furthermore, the support of Giganews is available 24 hours and 7 days a week by mail and
live chat. Usenet.nl and UseNeXT also provide support by mail and phone, and are unavailable only on Sundays.
Most of the other providers only offer support either per mail or by phone and only on weekdays.
Finally, there are differences in payment methods. Most options are offered by aEton. Here you can pay your bill
via bank transfer, direct debiting, credit card and PayPal. UseNeXT also provides those possibilities, except for
bank transfer. Typically a credit card is required and often the only option to pay. When it comes down to the
leading Usenet provider, UseNeXT might be your number one. You can easily describe UseNeXT as your allround-solution. It might not be the best choice in every division. So if you want to grasp the most out of your
Usenet access, UseNeXT might just be your way to go.

Which Usenet provider is the best for

you?
There are plenty of Usenet providers on the market. All providers claim to be the best, but they differ in many
ways. As a user, you want to get the very best of your provider, but you also want to pay as little as possible.
Therefore comparing the leading Usenet providers will show you, which provider offers unlimited download
speed to a reasonable price.
Looking at the leading providers, it doesnt matter much whether you choose Firstload, UsenetServer or any
other provider, because all of them offer unlimited download speed as long as you dont reach your download
limit where you will get throttled. Every company offers different packages, which mostly differ in costs and
download volume. UseNeXT, for example, provides their Smart-bundle for only 7,95 per month and 30 GB
download volume. Comparing this with Giganews more expensive Bronze-package for 9,99$ a month,
UseNeXT offers 20 GB unlimited downloads more. If you want limitless download volume in combination with
indefinite download speed, Giganews provides their Diamond-bundle for 29,99$ a month, which is rather
expensive. Looking at the price-performance ratio, UseNeXT offers the best allround-package, with 80 GB
download volume in full speed for only 9,95 monthly.
If you really cant decide which Usenet provider to choose yet, you might want to try a free-trial. Here,
UseNeXT, provides the best testing-options, as you can test their services for one whole month within which
you can download 15 GB with unlimited speed. As for other providers, for instance you can test every package
of Giganews for 14 days and download 10 GB, or download 15 GB from Firstload, but none of them can offer
you the complete package with as little effort as possible for you, as UseNeXT does.

What is Usenet?

Usenet is a worldwide discussion system distributed mainly (but not exclusively)

over the Internet (its formation actually predates the Internet). It consists of
thousands of publicly accessible so-called newsgroups each of which deals with a
specific topic. Everyone can post messages (or news articles, as they are called) to
any of these groups and read what others have posted. Everyone even can observing certain rules - create a new newsgroup dealing with a new topic.

Newsgroups are hierarchically structured. There are general top-level hierarchies

such as comp (for computing), soc (for social) etc. There is a special top-level
hierarchy alt (alternative) that is less regulated than the others with respect to
creating new newsgroups.
Hierarchy levels are separated by dots. For instance, comp.sys.next.software is a
newsgroup that deals with software for the NeXT computer system.
To make sure that only on-topic messages are posted, some newsgroups (by reader
decision) are moderated. Messages posted to moderated groups are not directly
distributed but automatically emailed to the moderator of the group, who decides if
the message will appear in the newsgroup.
If you are new to Usenet, you should read the
group news.announce.newusers which contains regular postings with information
for new users. Especially important are postings such as A Primer on How to Work
With the Usenet Community and Emily Postnews Answers Your Questions on
Netiquette that deal with the so-called Netiquette, i.e. common rules about how to
communicate with others via Usenet.
Usenet and the Internet

Usenet is a service on the Internet, as are Email, FTP or the WWW. However, it
has a different communication structure. With Email, two persons communicate
privately, just as they would on phone or with regular mail. WWW has a classical
vendor-consumer structure: there is one active information vendor who offers on
its (public or non-public) website information or services which can be used by
passive consumers.
In contrast, Usenet is public and egalitarian. There is no asymmetry between
vendors and consumers. Every participant in Usenet is both reader and contributor.
In this respect, Usenet, more than the other services, is what the Internet is really
about.
In keeping with this spirit of equality and free communication, the software which
sustains Usenet is is mostly free software.
Technical realization

The different communication structure of Usenet implies a different technical

realization. Email and WWW/FTP both have a point-to- point structure. For Email,
the Internet simply connects two computers:

Similarly, with WWW or FTP, many computers connect to one central computer the (WWW / FTP) server - with a point-to-point connection each:

This centralized model makes no sense for Usenet, where hundreds of thousands of
messages per day are posted from all parts of the world and read in all parts of the
world. A message posted from someone in New York should be accessible directly
in New York for all those people in New York who want to read it. Therefore,
Usenet is realized as a worldwide net of news servers which distribute the news
messages among each other:

(Note that this diagram is schematic and news servers are actually connected with
several other news servers in an arbitrary way, not in a cyclic structure.)

Someone wishing to post a message to Usenet connects with a news server close to
her. This news server receives the article and labels it with a message ID that is
guaranteed to be unique on the whole Usenet (by combining the (unique) Internet
address of the news server with a character sequence that the news server makes
sure is locally unique). Then the news server propagates the message to news
servers in its neighborhood, which propagate them to still more news servers, etc.
In this way, within hours the message is spread worldwide and can be read on all
news servers.
To prevent the message from being sent to news servers where it's already stored,
each news server enters its name in the Path: header of the message; a news server
won't propagate a message to another news server whose name is already included
in the Path: header of the message.
The software you use on your own computer to connect to a news server near you
and to read and post news messages is called a newsreader. As long as your own
computer is permanently connected to a local network that contains a news server,
this is no problem. However, if you have to connect to a news server of an Internet
service provider over a phone line, you can only read or post news messages as
long as you're online. Besides, because of the sheer number of newsgroups and
messages, an Internet service provider, who has to carry all newsgroups on its
server, may possibly store messages no longer than three or four days - maybe too
short a time for you to read all those you're interested in.
This is where PersonalINN comes in. It is a full-fledged news server for your own
computer. When you connect to the Internet, it will automatically exchange new
news articles in all newsgroups you're interested in with the news server of your
Internet service provider and will store the incoming articles on your local
computer for as long as you want. There, you (and everyone else on your local
computer or network) can read them with a newsreader of your/her choice, and
post articles yourself, without a connection to the Internet.