From: NeilBrown Date: Sat, 14 Aug 2010 08:04:47 +0000 (+1000) Subject: README update X-Git-Url: http://git.neil.brown.name/?a=commitdiff_plain;h=9fcc57d380fcb591a299fbf7f3a096c767788c95;p=LaFS.git README update --- diff --git a/README b/README index b2bc105..f95490e 100644 --- a/README +++ b/README @@ -4370,6 +4370,7 @@ TODO: watch -d 'awk -f checkseg /tmp/log; echo ====== ; grep -h -E "(blocked for more|BUG|SysRq : )" /var/tmp/lafs-logs/log* | sort | uniq -c ; echo === ; ls /var/tmp/lafs-logs/log* | wc -l' +watch -d 'echo ====== ; grep -h -E "(blocked for more|BUG|Busy inodes after|SysRq : )" /var/tmp/lafs-logs/log* | sort | uniq -c ; echo === ; ls /var/tmp/lafs-logs/log* | wc -l' Unclear on dirtying index blocks. @@ -4975,7 +4976,7 @@ DONE 14/ Review writepage and flush and make sure we flush often enough but DONE 14a/ use bio_add_page to write clusters. -14b/ Figure out what backing_dev to present for the filesystem. +DONE 14b/ Figure out what backing_dev to present for the filesystem. DONE 15/ The inode map file lost some credits. I think it losts a PinPending because it isn't locked properly. Don't clear PinPending if someone else might @@ -5000,10 +5001,11 @@ FIXED 15e/ setattr should queue an update for the inode metadata. 15g/ test directories with non-random sequential hash. -15h/ orphan deadlock +DONE 15h/ orphan deadlock lafs_run_orphans- lafs_orphan_release can block waiting for written in erase_dblock, but that won't complete until cleaner gets to run, but this is the cleaner blocked on orphans. + DONE 15i/ separate thread management from 'cleaner' name. @@ -5013,21 +5015,24 @@ DONE 15i/ separate thread management from 'cleaner' name. DONE 15k/ newblocks should probably be a count of segments. Review that. -15l/ make sure checkpoint_youth is decayed properly. Review youth decay. +DONE 15l/ make sure checkpoint_youth is decayed properly. Review youth decay. -15m/ consider combining .orphans and .cleaning lists. If something is an +DONE 15m/ consider combining .orphans and .cleaning lists. If something is an orphan, we probably don't want to clean it just now(?). -15n/ consider if lafs_pin_dblock should check for iolock. Maybe +DONE 15n/ consider if lafs_pin_dblock should check for iolock. Maybe iolock or PinPending (which must be set under iolock). + Just require PinPending and always get iolock_written for that + except in special cases. -15o/ Can there be async blocks when checkpoint starts? Could they +DONE 15o/ Can there be async blocks when checkpoint starts? Could they pin blocks in old phase? Do I need to check for them? -15p/ Review and remove the 'if cleaner is active then don't checkpoint just +DONE 15p/ Review and remove the 'if cleaner is active then don't checkpoint just yet' thing - or somehow avoid the yuckiness. -15q/ check checksums when reading cluster_header +DONE 15q/ check checksums when reading cluster_header for cleaner + This is already done! 15r/ consider further optimisation in cleaner to avoid lookups. @@ -5083,7 +5088,7 @@ DONE 15k/ newblocks should probably be a count of segments. Review that. 15am/ filesystem 'name' needs to be handle uniformly. -15an/ and we be sure 'b' will be non-null in delete_inode? +15an/ can we be sure 'b' will be non-null in delete_inode? 15ao/ determine what locking is needed to walk the children list in lafs_inode_handle_orphans. Probably the address_space private lock. @@ -5106,7 +5111,7 @@ DONE 15k/ newblocks should probably be a count of segments. Review that. 15av/ ditto for write errors? 15aw/ when lafs_incorporate makes a new block where the - only is Realloc, the new should be Realloc too. + old is Realloc, the new should be Realloc too. 15ax/ Think about what happens when we relocate a block in the orphan list (lafs_orphan_release), particularly @@ -5185,7 +5190,15 @@ DONE 15k/ newblocks should probably be a count of segments. Review that. - need multiple parallel write-clusters to allow parallel writes. - record tuning in state block: - max_segs + - use crc or something, not toy checksum (e.g. cluster - state already has) + - flags for inconsistencies found, at layout/fileset/file levels(?) + - policies of whether old or new data is allowed on each device + - policies of how much duplication of metadata is required + +15cc/ free any stray B_ASync block found in destroy_inode +15cd/ Some code assume a cluster header does not exceed 1 page. + Is this safe? Is in true? Is it enforced? 16/ Update locking.doc @@ -6413,3 +6426,38 @@ WritePhase - what is that all about? enough. How much precision is needed, and against what base? probsably mtime of last checkpoint from superblock. That should be not more than 2048 seconds ago, so 16 bits gets is 30msec... + +14Aug2010 + 15l - decay youth info. + Need to decay: + youth_next and checkpoint_youth in 'struct fs' + all blocks in youth files on storage + all scores in seg-tracker. + - not needed, they'll get updated in normal progress + and being wrong for a while is no cost. + ensure correct youth is stored in lafs_free_get + check little-endian conversion of all youth accesses + + checkpoint_youth only used by thread, so no locking needed + youth_next protected by fs->lock + + 15m - share orphans and cleaning list_heads in datablock + It certainly is possible to clean an orphan but it is very unlikely + as it will have changed recently, or be changing soon. + The cleaner could just dirty any B_Orphan it finds. + But if orphan finds a block on the list, it must be careful... + I guess when cleaner drops a cleaning ref, it should check if the block + is an orphan, and re-queue if it is. + + 15o - async blocks just have an extra refcount. + This could: + - keep PinPending set + - keep an index block pinned - will phase-flip + - keep ->parent link + not not get in the way of a checkpoint. + + Should we clear any that we find though? + Normally async is only used by cleaner, orphan processing, or segscan + So it should all be finished when we do a checkpoint. + + So if checkpoint, or release_page, finds an async block, drop it. diff --git a/clean.c b/clean.c index 1884542..7c6b6dd 100644 --- a/clean.c +++ b/clean.c @@ -168,7 +168,6 @@ static int try_clean(struct fs *fs, struct toclean *tc) goto out; } BUG_ON(err); - // FIXME check checksum of the cluster_header tc->ch = page_address(tc->chead); if (memcmp(tc->ch->idtag, "LaFSHead", 8) != 0) goto bad_header;