Money, mirroring

[Micah Yoder <LYoder@cyberis.net>]

"Aaron D. Turner" wrote:

> The difference is, do you want www.linuxkb.org to mean something, or do
> you want a distribution method simular to the LDP?  I strongly believe
> that modeling the LKB as a destination site along the lines of
> www.kernel.org, www.linuxhq.com, etc is a far better, less confusing, and
> more effective experiance for the average user.  People want one URL for a
> site, not a bunch of URL's that don't seem to relate, especially when
> anyone of them may not be up to date or even a mirror anymore.
> 
> Also, from my experiance, the "master" site gets 90% of the hits and the
> mirrors get the scraps.  This is just because other webmasters are too
> lazy to list multiple URL's for a site.  They just list the master.
> User's are too lazy to "pick the nearest site", not to mention the fact
> that "nearest" in Internet terms is a confusing proposition at best.

Here's what I think we should do:  have www.linuxkb.com be a "dummy"
site with some basic information and links to mirrors, all in static
HTML.  All mirrors should be accessible to us, and be under the
linuxkb.org domain, and have geographic information.  Like, Navisite
would be us-ca.linuxkb.org, Germany could have a de.linuxkb.org, Mexico
could have mx.linuxkb.org, etc.

We would need an organization to give us hardware and bandwidth for each
mirror.  I really think that would be easy to do.  Their logos would be
displayed on their mirror.

Geographic diversity is our friend.  Yes, it will be a pain to set up,
but that's what we're here for.

> The logo's along with the statement was part of the deal.  When I was
> leading our group I felt that it was important to get a dedicated box with
> excellent connectivity.  This seemed like a good/quick/easy way to
> accomplish that.  It was also an effective motivator for us as group.

Yes... that kind of thing is good for us and for them.  Hopefully we can
give them some business.  It's better than cash for us (no legal
hassles) and good publicity for them.

> the LDP!)  And even if we don't mirror/host another site, maybe we donate
> $$$ to support a penguin at a zoo.  Maybe we pay someone to write an OSS
> driver for USB or something.  The fact is that there's a lot of good we
> could do with the $$$, beyond purchasing hardware for the site.

Whooaaahhh....  *That*'s going a bit beyond our "charter".  :-)  Sure,
money can be used for all kinds of good, but it was never a purpose of
ours to make money, or even to do good.  :-)  Of course, our
contribution will be a good place to find information on Linux.  That's
what we're here to contribute, and I don't think we need to try to
accomplish more.

> While I'll agree serving/mirroring the static HTML content is a breeze,
> that is only a minor part of the overall utilization of the box.  The
> CPU/disk hog won't be the static HTML, MySQL, Apache, or even the Perl
> code we write, it's going to be Ht://dig.
> 
> Ht://dig uses a DB2 database which will be virutually unmirrorable (the DB
> is generated via a spyder, hence incremental updates aren't likely to
> work, even if we found a way to do it with DB2).  The DB will be 4 times
> the size of the static HTML due to the fuzzly logic extentions.  And
> realize that 90+% of users aren't going to "browse" the site.  They're
> going to type a key word or two and click "search".  If we're lucky, for
> every 5 pages served, only one will be a Ht://dig results page.

I don't know exactly how Ht://dig works, but I have a hard time
believing it's unmirrorable.  It's open source, we can hack it.  We
would just need to write a program to do the updates, and run a daemon
on the mirrors to listen for them.  It would work something like this:

When a new article is added, a new HTML page is generated, some SQL will
happen to get it into the RDBMS, and ht://dig will need to know about
it.  We'd probably use a special section of the main site,
www.linuxkb.org, to add stuff.  It would then generate all the info
needed and send it to all the mirrors, possibly by E-mail, maybe some
other way.  The mirrors would run the same SQL command to get it into
their databases, and ht://dig would be notified to add the page.

Does ht://dig have a way to add a page other than a spider search?  I
hate spiders...  (the web kind...the 8 legged kind are kinda neat)

This has *got* to be do-able.

We really do need to decide on something soon.  We won't need mirrors
for a while, but we do need to get the code in place to run them, so we
can put them up easily when we need them and get donations.

> The only reason to mirror the site is if we can scale the search engine;
> anything else is meaningless.  Having the boxes at a single location

We need to scale the search engine AND provide faster response for
overseas users.

> behind a Local Director which gives the illusion of one big box makes
> scaling the system a breeze.  Otherwise you have to convince others to run
> and maintain a CPU/disk intensive search engine.  At that point it's no
> longer a simple mirror site, it's almost a full blown replica and that's a
> lot harder sell to volunteers.  And then you still have the failing

I think we'd still have control of everything.  Just like the Navisite,
VAR deal, they'd give us the box and bandwidth and we'd do the rest.  It
would *not* share a CPU with something else...someone would need to give
us a box whole-hog.

> proposition getting people to actually use the mirrors.  The result is
> www.linuxkb.org is a massively overloaded site in 6-12 months no matter
> how many satellite mirrors we have.

Not if we just make it a static set of links.  :-)

> One last thing.  There is an advantage with corporate sponsorships-
> immediate visibility outside the Linux community.  VA Research for
> example, when we go live is going to do a press release on NewsWire about
> us.  This means that we now have visibility to traditional media and those
> who aren't connected to the Linux community.  Think about what that means
> in terms of not just the number of visitors, but the diversity, and how
> that helps promote Linux and pro-Linux compaines such as VA Research.

That's great!  They can do that because they're contributing hardware --
we don't need to ask for cash.

> They can help us and we can help them and by helping each other we help
> Linux.  As long as we keep that the purpose, money isn't a dividing issue
> anymore, it becomes a tool.

Right, but it hasn't been proven that we need it yet.  My opnion of
money is this:  We don't need it now, period.  We might in the future,
and if that's the case, THEN we can go after it.  I really hope it
doesn't have to be with banner ads though.  That would cheapen the site
a bit.  This really should look professional.