[Fwd: Re: Lab to sample Linux for weapons work]

[Bari Ari]

-------- Original Message --------
Subject: Re: Lab to sample Linux for weapons work
Date: Thu, 26 Sep 2002 15:06:56 -0400
From: Velocet 
To: [e-mail]
References:

On Thu, Sep 26, 2002 at 03:49:39PM +0200, Eugen Leitl's all...
> There's a major difference compared with Livermore system, though: The Los 
> Alamos machine has no hard drives. Instead, each computer fires up using 
> software pulled over the network with the assistance of software called 
> LinuxBIOS developed by LANL programmer Ron Minnich and others. LinuxBIOS 
> also dramatically speeds the startup process to about two seconds, said 
> Jason Lowry, Linux NetworX's product manager for cluster management tools.

And what filesystem are they using for this? Surely its not NFS!
Must be some sort of cluster filesystem no? Anyone know? Something open
sourced? What decent clustre filesystems are out there?

/kc
>
> Shunning hard drives cuts cost and power consumption, but more 
> importantly, it improves reliability, Roundy said.
>
> "If you think about what things are going to fail in a system, it's the 
> hard disk or fan or power supply or something with moving parts," Roundy 
> said.
>
> Linux NetworX could benefit greatly from convincing the Los Alamos and 
> Livermore labs that Linux clusters are worthwhile. The labs are funded by 
> the Energy Department's Advanced Simulation and Computing program, which 
> has spent hundreds of millions of dollars to advance supercomputing using 
> machines made of comparatively inexpensive components.
>
> The DOE program has underwritten many of the world's fastest computers, 
> according to university researchers who monitor raw calculation speed at 
> the Top500 organization. The program has underwritten Nos. 2, 6, 7, 9, 11, 
> and 15 on the most recent ranking.
>
> Beowulf mailing list, [e-mail]
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

[Ronald G Minnich]

On Thu, 26 Sep 2002, Bari Ari wrote:
> And what filesystem are they using for this? Surely its not NFS!
> Must be some sort of cluster filesystem no? Anyone know? Something open
> sourced? What decent clustre filesystems are out there?

NFS is dead for clusters.

We are targeting three possible systems, each having a different set of
advantages:
1- panasas (http://www.panasas.com)
2- lustre (http://www.lustre.org)
3- v9fs (http://v9fs.sourceforge.net), from yours truly

ron

[Gregg C Levine]

Hello from Gregg C Levine
Ron, about the v9fs project. I am impressed. It looks like a good
project. I might even try the code in any of my projects. So? What did
kill NFS for clusters? Just for fun, try the v9fs in the first cluster
that starts life at your shop.
-------------------
Gregg C Levine [e-mail]
------------------------------------------------------------
"The Force will be with you...Always." Obi-Wan Kenobi
"Use the Force, Luke."  Obi-Wan Kenobi
(This company dedicates this E-Mail to General Obi-Wan Kenobi )
(This company dedicates this E-Mail to Master Yoda )
> -----Original Message-----
> From: [e-mail] [mailto:linuxbios-
> [e-mail]] On Behalf Of Ronald G Minnich
> Sent: Thursday, September 26, 2002 10:37 PM
> To: Bari Ari
> Cc: LinuxBios
> Subject: Re: [Fwd: Re: Lab to sample Linux for weapons work]
>
> On Thu, 26 Sep 2002, Bari Ari wrote:
> > And what filesystem are they using for this? Surely its not NFS!
> > Must be some sort of cluster filesystem no? Anyone know? Something

open
> > sourced? What decent clustre filesystems are out there?
>
> NFS is dead for clusters.
>
> We are targeting three possible systems, each having a different set

of
> advantages:
> 1- panasas (http://www.panasas.com)
> 2- lustre (http://www.lustre.org)
> 3- v9fs (http://v9fs.sourceforge.net), from yours truly
>
> ron
>
> Linuxbios mailing list
> [e-mail]
> http://www.clustermatic.org/mailman/listinfo/linuxbios

[Ronald G Minnich]

On Thu, 26 Sep 2002, Gregg C Levine wrote:
> Ron, about the v9fs project. I am impressed. It looks like a good
> project. I might even try the code in any of my projects. So? What did
> kill NFS for clusters? Just for fun, try the v9fs in the first cluster
> that starts life at your shop.

thanks.

NFS has a fundamental assumptions that causes endless trouble.  The
assumption is statelessness. This means that servers and clients can
restart any time. But for performance, true stateless operation is never
practiced, and then you have to add timeouts for inode data, and the whole
thing starts to look like a giant collection of rubber bands, glue, and
paper clips.

Basically if you look hard at NFS all the bad problems flow from this
assumption. This has been written up to death over the last decade, and
we're finally able to move away from NFS as it does not seem fixable for
clusters.

We are testing v9fs on clusters now. It works and with 2.4.19 and its own
private name space support it is really nice.

ron