Find in a given directory, if there exist 2 files that have exactly the same content irrespective of their...

This question already has an answer here:

Can I get the opposite of `diff -q` — matching identical files without printing their contents

4 answers

Find duplicate files

7 answers

if the problem could be solved using a series of commands on the command line, it would be better for me than writing a script

edited Jan 28 at 18:06

Rui F Ribeiro

39.9k1479135

asked Jan 28 at 18:00

jospin deek

marked as duplicate by Rui F Ribeiro, Christopher, RalfFriedl, Jeff Schaller, G-Man Jan 28 at 19:01

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

Hi and welcome to Unix Stack Exchange. I think to do this, you would need to use the diff command. You would probably have to construct a for loop to compare each file in the directory to every other. Could do it with a fairly simple script.

– Time4Tea
Jan 28 at 18:05

add a comment |

This question already has an answer here:

Can I get the opposite of `diff -q` — matching identical files without printing their contents

4 answers

Find duplicate files

7 answers

if the problem could be solved using a series of commands on the command line, it would be better for me than writing a script

edited Jan 28 at 18:06

Rui F Ribeiro

39.9k1479135

asked Jan 28 at 18:00

jospin deek

marked as duplicate by Rui F Ribeiro, Christopher, RalfFriedl, Jeff Schaller, G-Man Jan 28 at 19:01

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

Hi and welcome to Unix Stack Exchange. I think to do this, you would need to use the diff command. You would probably have to construct a for loop to compare each file in the directory to every other. Could do it with a fairly simple script.

– Time4Tea
Jan 28 at 18:05

add a comment |

This question already has an answer here:

Can I get the opposite of `diff -q` — matching identical files without printing their contents

4 answers

Find duplicate files

7 answers

if the problem could be solved using a series of commands on the command line, it would be better for me than writing a script

edited Jan 28 at 18:06

Rui F Ribeiro

39.9k1479135

asked Jan 28 at 18:00

jospin deek

This question already has an answer here:

Can I get the opposite of `diff -q` — matching identical files without printing their contents

4 answers

Find duplicate files

7 answers

if the problem could be solved using a series of commands on the command line, it would be better for me than writing a script

This question already has an answer here:

Can I get the opposite of `diff -q` — matching identical files without printing their contents

4 answers

Find duplicate files

7 answers

bash shell-script command-line

edited Jan 28 at 18:06

Rui F Ribeiro

39.9k1479135

asked Jan 28 at 18:00

jospin deek

edited Jan 28 at 18:06

Rui F Ribeiro

39.9k1479135

asked Jan 28 at 18:00

jospin deek

edited Jan 28 at 18:06

Rui F Ribeiro

39.9k1479135

edited Jan 28 at 18:06

Rui F Ribeiro

39.9k1479135

edited Jan 28 at 18:06

Rui F Ribeiro

39.9k1479135

asked Jan 28 at 18:00

jospin deek

asked Jan 28 at 18:00

jospin deek

asked Jan 28 at 18:00

jospin deek

marked as duplicate by Rui F Ribeiro, Christopher, RalfFriedl, Jeff Schaller, G-Man Jan 28 at 19:01

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

marked as duplicate by Rui F Ribeiro, Christopher, RalfFriedl, Jeff Schaller, G-Man Jan 28 at 19:01

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

Hi and welcome to Unix Stack Exchange. I think to do this, you would need to use the diff command. You would probably have to construct a for loop to compare each file in the directory to every other. Could do it with a fairly simple script.

– Time4Tea
Jan 28 at 18:05

add a comment |

Hi and welcome to Unix Stack Exchange. I think to do this, you would need to use the diff command. You would probably have to construct a for loop to compare each file in the directory to every other. Could do it with a fairly simple script.

– Time4Tea
Jan 28 at 18:05

Hi and welcome to Unix Stack Exchange. I think to do this, you would need to use the diff command. You would probably have to construct a for loop to compare each file in the directory to every other. Could do it with a fairly simple script.

– Time4Tea
Jan 28 at 18:05

add a comment |

2 Answers
2

active

oldest

votes

for x in *; do for y in *; do [ "$x" = "$y" ] && continue; cmp -s "$x" "$y" && echo Same: "$x" and "$y"; done; done|head -1

Or, broken up a bit for readability:

for x in *

do 

  for y in *

  do 

    [ "$x" = "$y" ] && continue

    cmp -s "$x" "$y" && echo Same: "$x" and "$y"

  done

done | head -1

The head is just to keep the mirror reporting down ("a = b" and "b = a").

answered Jan 28 at 18:11

Jeff Schaller

41k1056131

thank you, that was very helpful

– jospin deek
Jan 28 at 18:21

add a comment |

find . -type f -exec md5sum "{}" ; | awk 'seen[$1] { print "Duplicate file "$2" with hash "$1" at "seen[$1]" } ! seen[$1] {seen[$1]=$2}'

edited Jan 28 at 20:17

answered Jan 28 at 18:22

DopeGhoti

45.3k55988

It says "Duplicate file xxx with hash yyy". That's right. But duplicate to what file? But nicely scripted!

– Freddy
Jan 28 at 18:55

Tweaked to store the first filename found in the array for reference in the output.

– DopeGhoti
Jan 28 at 20:17

add a comment |

2 Answers
2

active

oldest

votes

2 Answers
2

active

oldest

votes

for x in *; do for y in *; do [ "$x" = "$y" ] && continue; cmp -s "$x" "$y" && echo Same: "$x" and "$y"; done; done|head -1

Or, broken up a bit for readability:

for x in *

do 

  for y in *

  do 

    [ "$x" = "$y" ] && continue

    cmp -s "$x" "$y" && echo Same: "$x" and "$y"

  done

done | head -1

The head is just to keep the mirror reporting down ("a = b" and "b = a").

answered Jan 28 at 18:11

Jeff Schaller

41k1056131

thank you, that was very helpful

– jospin deek
Jan 28 at 18:21

add a comment |

for x in *; do for y in *; do [ "$x" = "$y" ] && continue; cmp -s "$x" "$y" && echo Same: "$x" and "$y"; done; done|head -1

Or, broken up a bit for readability:

for x in *

do 

  for y in *

  do 

    [ "$x" = "$y" ] && continue

    cmp -s "$x" "$y" && echo Same: "$x" and "$y"

  done

done | head -1

The head is just to keep the mirror reporting down ("a = b" and "b = a").

answered Jan 28 at 18:11

Jeff Schaller

41k1056131

thank you, that was very helpful

– jospin deek
Jan 28 at 18:21

add a comment |

for x in *; do for y in *; do [ "$x" = "$y" ] && continue; cmp -s "$x" "$y" && echo Same: "$x" and "$y"; done; done|head -1

Or, broken up a bit for readability:

for x in *

do 

  for y in *

  do 

    [ "$x" = "$y" ] && continue

    cmp -s "$x" "$y" && echo Same: "$x" and "$y"

  done

done | head -1

The head is just to keep the mirror reporting down ("a = b" and "b = a").

answered Jan 28 at 18:11

Jeff Schaller

41k1056131

for x in *; do for y in *; do [ "$x" = "$y" ] && continue; cmp -s "$x" "$y" && echo Same: "$x" and "$y"; done; done|head -1

Or, broken up a bit for readability:

for x in *

do 

  for y in *

  do 

    [ "$x" = "$y" ] && continue

    cmp -s "$x" "$y" && echo Same: "$x" and "$y"

  done

done | head -1

The head is just to keep the mirror reporting down ("a = b" and "b = a").

answered Jan 28 at 18:11

Jeff Schaller

41k1056131

answered Jan 28 at 18:11

Jeff Schaller

41k1056131

answered Jan 28 at 18:11

Jeff Schaller

41k1056131

answered Jan 28 at 18:11

Jeff Schaller

41k1056131

thank you, that was very helpful

– jospin deek
Jan 28 at 18:21

add a comment |

thank you, that was very helpful

– jospin deek
Jan 28 at 18:21

thank you, that was very helpful

– jospin deek
Jan 28 at 18:21

add a comment |

find . -type f -exec md5sum "{}" ; | awk 'seen[$1] { print "Duplicate file "$2" with hash "$1" at "seen[$1]" } ! seen[$1] {seen[$1]=$2}'

edited Jan 28 at 20:17

answered Jan 28 at 18:22

DopeGhoti

45.3k55988

It says "Duplicate file xxx with hash yyy". That's right. But duplicate to what file? But nicely scripted!

– Freddy
Jan 28 at 18:55

Tweaked to store the first filename found in the array for reference in the output.

– DopeGhoti
Jan 28 at 20:17

add a comment |

find . -type f -exec md5sum "{}" ; | awk 'seen[$1] { print "Duplicate file "$2" with hash "$1" at "seen[$1]" } ! seen[$1] {seen[$1]=$2}'

edited Jan 28 at 20:17

answered Jan 28 at 18:22

DopeGhoti

45.3k55988

It says "Duplicate file xxx with hash yyy". That's right. But duplicate to what file? But nicely scripted!

– Freddy
Jan 28 at 18:55

Tweaked to store the first filename found in the array for reference in the output.

– DopeGhoti
Jan 28 at 20:17

add a comment |

find . -type f -exec md5sum "{}" ; | awk 'seen[$1] { print "Duplicate file "$2" with hash "$1" at "seen[$1]" } ! seen[$1] {seen[$1]=$2}'

edited Jan 28 at 20:17

answered Jan 28 at 18:22

DopeGhoti

45.3k55988

find . -type f -exec md5sum "{}" ; | awk 'seen[$1] { print "Duplicate file "$2" with hash "$1" at "seen[$1]" } ! seen[$1] {seen[$1]=$2}'

edited Jan 28 at 20:17

answered Jan 28 at 18:22

DopeGhoti

45.3k55988

edited Jan 28 at 20:17

answered Jan 28 at 18:22

DopeGhoti

45.3k55988

answered Jan 28 at 18:22

DopeGhoti

45.3k55988

answered Jan 28 at 18:22

DopeGhoti

45.3k55988

It says "Duplicate file xxx with hash yyy". That's right. But duplicate to what file? But nicely scripted!

– Freddy
Jan 28 at 18:55

Tweaked to store the first filename found in the array for reference in the output.

– DopeGhoti
Jan 28 at 20:17

add a comment |

It says "Duplicate file xxx with hash yyy". That's right. But duplicate to what file? But nicely scripted!

– Freddy
Jan 28 at 18:55

Tweaked to store the first filename found in the array for reference in the output.

– DopeGhoti
Jan 28 at 20:17

It says "Duplicate file xxx with hash yyy". That's right. But duplicate to what file? But nicely scripted!

– Freddy
Jan 28 at 18:55

Tweaked to store the first filename found in the array for reference in the output.

– DopeGhoti
Jan 28 at 20:17

add a comment |

This page is only for reference, If you need detailed information, please check here

搜尋此網誌

Ytdyklly