File system related stuff

What is an inode?

inode (index node)
inode - Wikipedia, the free encyclopedia
An inode is a data structure on a traditional Unix-style file system such as UFS or ext3. An inode stores basic information about a regular file, directory, or other file system object. A POSIX filesystem includes things like size in bytes, deviceid,userid,guid,filemode,timestamps,linkcounts,pointers to disk blocks.
File names and directory implications:

inodes do not contain file names, only other file metadata.
Unix directories are lists of association structures, each of which contains one filename and one inode number.
The file system driver must search a directory looking for a particular filename and then convert the filename to the correct corresponding inode number.

How do I see file inode number? ls -i

difference between a symbolic link and a hard link?

unix - What is the difference between a symbolic link and a hard link? - Stack Overflow

btrfs

A short history of btrfs [LWN.net]
Conversion from Ext3 - btrfs Wiki

Ext

Ext2Fsd Project » About Ext2Fsd - Ext2Fsd is an ext2 file system driver for Windows 2000, XP, Vista.
Ext4 Howto - Ext4

ZFS

ZFS on Linux zfsonlinux/zfs
The Z File System (ZFS)
Samba with ZFS
Using ZFS to add unlimited snapshots to NTFS filesystems - Greg Beary's blog
Aaron Toponce : ZFS Administration, Part XV- iSCSI, NFS and Samba
Aaron Toponce : ZFS Administration, Part XIV- ZVOLS - A ZVOL is a “ZFS volume” that has been exported to the system as a block device.
Aaron Toponce : ZFS Administration, Part XIII- Sending and Receiving Filesystems
Aaron Toponce : ZFS Administration, Part XII- Snapshots and Clones

NILFS

NILFS: A File System to Make SSDs Scream | Linux Magazine

Hadoop || HDFS || Map-Reduce || HBASE

master NameNode/JobTracker, slave DataNode/TaskTracker.
NameNode is in memory; secondary NameNode is write to disk checkpoints.
data block redundancy provided by replication across nodes. hdfs-site.xml - dfs.replication
heart beats to namedname, respawned data from remaining datanodes; self-healing.
FrontPage - Hadoop Wiki
Get to Know Hadoop Filesystems - O'Reilly Answers
Introduction to Hadoop: Real-World Hadoop Clusters and Applications | Dr Dobb's
Welcome to Apache Flume — Apache Flume
Acquiring Big Data Using Apache Flume | Dr Dobb's
Hadoop Default Ports Quick Reference | Apache Hadoop for the Enterprise | Cloudera
PUMA MapReduce Datasets Download
Hadoop Installation Tutorial | Computing systems | Highly Scalable Systems
mrcc – A Distributed C Compiler System on MapReduce | Computing systems | Highly Scalable Systems
hadoop fs -ls|-mkdir|-copyFromLocal|-copyToLocal|-moveToLocal|-rm|-tail|-chmod|-setrep -w 4 -R /recursive/replicationfactor
SQL Server Connector for Hadoop - TechNet Articles - United States (English) - TechNet Wiki
CDH 5.1.2 - CDH is Cloudera's open source software distribution and consists of Apache Hadoop and additional key open source projects to ensure you get the most out of Hadoop and your data.
HBASE - distributed, versioned, column oriented database.

MooseFS

Features, Architecture and Requirements :: MooseFS network file system - Moose FS - FUSE based. MooseFS is a fault tolerant, network distributed file system. single master, multiple chunk servers, with metadata backup for master restore. chunk servers communicate with each other to replicate data in order to achieve the "goal" number of copies of a file on different machines.
thatch45/mfs-failover - Failover scripts for MooseFS

integrity tools, checksums, dup finders

fswatch - file system checksum checker
Akiscode - How to get a SHA-1 Hash of a directory - python implementation.
Compress dd backups with sha1sum checksum | rootninja
Checksumming Files to Find Bit-Rot | Linux Magazine - checksum with extended file attributes, python implementation.

Linux software raid

Mdadm Cheat Sheet - MDLog:/sysadmin - cat /proc/mdstat
Increase the speed of Linux Software RAID reconstruction - MDLog:/sysadmin - minimum _guaranteed_ reconstruction speed: 1000 KB/sec/disc.,echo 50000 >/proc/sys/dev/raid/speed_limit_min,watch cat /proc/mdstat

gluster

Write once, read everywhere — Gluster
Gluster Site Relaunches | Gluster Community Website
Docker + GlusterFS = Dockit (An easy way of GlusterFS deployment with Docker) | Gluster Community Website humblec/dockit
The Technical Blog of James | Technical articles, writeups and discussion! purpleidea/puppet-gluster
my gluster setup, described | The Technical Blog of James

CRC: The C Stands for Confusion | Dr Dobb's
mogilefs - MogileFS distributed filesystem - Google Project Hosting

Created: 2010-03-14 22:45:54 Modified: 2014-09-10 19:37:16