{"id":16,"date":"2012-08-10T21:31:39","date_gmt":"2012-08-10T21:31:39","guid":{"rendered":"http:\/\/www.notnull.ro\/log\/?p=16"},"modified":"2014-01-15T23:03:15","modified_gmt":"2014-01-15T23:03:15","slug":"find-dead-links-on-unix-with-perl","status":"publish","type":"post","link":"https:\/\/notnull.ro\/log\/find-dead-links-on-unix-with-perl\/","title":{"rendered":"Find Dead Links on Unix with Perl"},"content":{"rendered":"<p>Very often on unix environment we have to deal with symlinks (aka soft links &#8211; reference to other files).<\/p>\n<p>Because of different reason the referenced file may be moved or removed, but the\u00a0 symlink still points to the old file.<\/p>\n<p>\nSuch situations can lead to unexpected errors: e.g if your symlink points to an executable and the referenced file was deleted then you will get &#8220;Permission denied&#8221; message, and you will ask yourself: why it does not work ,couple of days ago it was just fine; and the file is there ?!.<\/p>\n<p>Because of this few years ago I&#8217;ve made a small script to look for dead links.<br \/>\nUsually I start this script during weekend, and Monday morning I&#8217;m just looking on the reports.<br \/>\n<!--more--><br \/>\nI want it to run on different Unix flavors, so I&#8217;ve choose something portable: <del>JAVA<\/del> Perl .\n<\/p>\n<p>\nWhat is doing this script:<\/p>\n<ol>\n<li>get all the files\/dirs\/links from the directory passed as parameter<\/li>\n<li>normal files are just ignored<\/li>\n<li>for each link found in the start dir check if the referenced object exist.\n<ul>\n<li>if it does not exist put the link in a hash that will store all the dead links<\/li>\n<li>if the referenced object is a directory then consider this one as a new input for step #1<\/li>\n<\/ul>\n<\/li>\n<li>For all the directories found in the starting directory apply again step #1<\/li>\n<\/ol>\n<p>In a few words: it is a recursive search down into the hierarchy.\n<\/p>\n<p>\nThere is just one interesting thing, because of the symlinks there are chances to have cycles in the files structure.<br \/>\nAnd it means that with this algorithm we may enter into a never ending loop.\n<\/p>\n<p>\nThe solution is to keep track of all the already scanned directories, and when they are found 2<sup>nd<\/sup> time, they will be ignored &#8211; this can be done using other hash (aka associative array).<br \/>\nWe will not keep in this hash the name\/path of the directory because we can also have hard links of the files (to paths pointing to the same &#8220;content&#8221;), and because of this we need some info which unique identifies the files\/directories content.<br \/>\nAnd this info is provided by <em>stat($file)<\/em> Perl routine.<br \/>\nstat() return a lot of things, but we are interested only in first 2 elements: device number of files system and the inode number.<br \/>\nThey unique identify the files.\n<\/p>\n<p>\n<a href='http:\/\/www.notnull.ro\/log\/wp-content\/uploads\/2012\/08\/dead_links.zip'>Here is the dead_links.pl<\/a> , and bellow is the code (just copy->paste the code if the download link has problems)\n<\/p>\n<p><a href=\"https:\/\/notnull.ro\/log\/feedback\/\" title=\"We believe in feedback\" target=\"_blank\">Feedback<\/a> is appreciated, comments appreciated too.<\/p>\n<pre>\r\n\r\n#!\/usr\/local\/bin\/perl\r\n# search for dead symlinks into hierarchy\r\n# author marcel_preda[at]yahoo.com\r\n\r\n\r\nuse strict ;\r\n# a global hash with  dead links\r\nmy %DeadLinks = ();\r\n\r\n# a global with the visited dev-inodes \r\nmy %DevInodes = ();\r\n\r\n\r\n# print a short help\r\n# every `pro' script should have something like this\r\nsub printUse\r\n{\r\n\tprint &lt;&lt;EOF;\r\n\\nUse\t`$0 [Options]'\r\n\t`$0 [directoryName]'\r\n\tValid options:\r\n\t\t-h, --help print this help ;\r\n\r\nEOF\r\n\texit 0;\r\n}\r\n\r\n\r\n# search (and print) in a subdirectory\r\nsub searchLinks\r\n{\r\n# the global %DeadLinks, %DevInodes variables will be used in this subroutine\r\n\r\n\tmy($file,@files,@files2);\r\n\tmy ($dev, $inode) = stat($_[0]);\r\n\t# $dev:$inode identifies the file \r\n\tif ($DevInodes{\"$dev:$inode\"}){\r\n\t\t# directory was already visited, probable a cyclic link or hard link\r\n\t\t# print \"VISITED: \", $_[0], \" \\n\";\r\n\t\treturn; \r\n\t};\r\n\r\n\t#mark dir as visited to not scan it next time\r\n\t$DevInodes{\"$dev:$inode\"} = 1;\r\n\r\n\tif (opendir(DIR_HANDLE,$_[0])){\r\n\t\twhile ($file=readdir(DIR_HANDLE)) {\r\n\t\t\tpush(@files,\"$_[0]\/$file\");\r\n\t\t};\r\n\t\tclosedir(DIR_HANDLE);\r\n\t\t@files2=grep((!\/\\.\\.?$\/),@files);\t#skip `.' and `..' dirs\r\n\t\tfor $file (@files2){\r\n\t\t\t# if link, check if it is not a dead one\r\n\t\t\tif ( -l $file ) {\r\n\t\t\t\t($dev, $inode) = stat $file;\r\n\t\t\t\tif ((! $dev) && (! $inode)){\r\n\t\t\t\t\t$DeadLinks{$file} = readlink $file;\r\n\t\t\t\t};\r\n\t\t\t};\r\n\t\t\t\t\r\n\t\t\t# if is directory call again  the subrutine\r\n           \tsearchLinks($file) if (-d $file) ;\r\n\t\t}\r\n\t};\r\n}\r\n\r\n#main\r\nmy $DIR=$ARGV[0];\r\n\r\n# print help\r\nwhile ( $_ = shift ){\r\n\t(\/^-h$\/ || \/^-+help$\/ ) && printUse(); \r\n};\r\n\r\n# no dir provided, use the $PWD\r\nunless ($DIR) {$DIR='.';};\r\n\r\nsearchLinks($DIR);\r\n\r\n# Report dead links\r\nmy $link;\r\nforeach $link (keys %DeadLinks){\r\n    print 'DEAD LINK',  \"\\t $link\\t-&gt;\\t\",  $DeadLinks{$link}, \"\\n\";\r\n};\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Very often on unix environment we have to deal with symlinks (aka soft links &#8211; reference to other files). Because of different reason the referenced file may be moved or removed, but the\u00a0 symlink still points to the old file. Such situations can lead to unexpected errors: e.g if your symlink points to an executable <a href='https:\/\/notnull.ro\/log\/find-dead-links-on-unix-with-perl\/' class='excerpt-more'>&#8230; Read Full Article<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6,5,7],"tags":[11,10,12,14,15,16,13],"class_list":["post-16","post","type-post","status-publish","format-standard","hentry","category-perl","category-programming","category-unix","tag-dead","tag-find","tag-links","tag-linux-2","tag-perl-2","tag-recursive","tag-unix-2","category-6-id","category-5-id","category-7-id","post-seq-1","post-parity-odd","meta-position-corners","fix"],"_links":{"self":[{"href":"https:\/\/notnull.ro\/log\/wp-json\/wp\/v2\/posts\/16","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/notnull.ro\/log\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/notnull.ro\/log\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/notnull.ro\/log\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/notnull.ro\/log\/wp-json\/wp\/v2\/comments?post=16"}],"version-history":[{"count":21,"href":"https:\/\/notnull.ro\/log\/wp-json\/wp\/v2\/posts\/16\/revisions"}],"predecessor-version":[{"id":273,"href":"https:\/\/notnull.ro\/log\/wp-json\/wp\/v2\/posts\/16\/revisions\/273"}],"wp:attachment":[{"href":"https:\/\/notnull.ro\/log\/wp-json\/wp\/v2\/media?parent=16"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/notnull.ro\/log\/wp-json\/wp\/v2\/categories?post=16"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/notnull.ro\/log\/wp-json\/wp\/v2\/tags?post=16"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}