Also forgot to note:
1012024699 total lines in the file

full base

magnet:?xt=urn:btih:7ffbcd8cee06aba2ce6561688cf68ce2addca0a3&dn=BreachCompilation&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80&tr=udp%3A%2F%2Ftracker.leechers-paradise.org%3A6969&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969&tr=udp%3A%2F%2Fglotorrents.pw%3A6969&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337

modInfo, is it txt file?

Welp, looks like this is not sorted, so not a unique list. :(

1. sort with the number of cores you have available:

LC_ALL=C sort --parallel=8 -u breachcompilation.txt -o breachcompilation.sorted.txt

2. remove leading spaces:

sed -i -e's/^s*//' breachcompilation.sorted.txt

Check out this thread for more info:

There is about 38% of the list that is unique (with sort & uniq I get 384149228 lines).
And a lot are not only password but email:password

There are quite a few lines like this in the list as well:

(null)resettingpassword..retry#1
(null)resettingpassword..retry#2

je n'arrive pas a la télécharger y'aurais pas un autre liens ?

@folet1992 il te faut bittorrent ou autre client de torrents puis utilise le magnet sur le post original.

grep -rohP '(?<=:).*$' sort uniq -c sort -nr > breachcompilation_index.txt
cat breachcompilation_index.txt cut -b 9- > breachcompilation.txt

uncompressed filesize:
6.7G Dec 20 19:35 breachcompilation_index.txt
3.8G Dec 20 23:14 breachcompilation.txt Best motion capture software.

I have a file with a password count and one without..

It seems just over half this file is wasted space. The uniq command only removed duplicates that are directly following each other.

3.8G breachcompilation_sorted.txt
9.0G breachcompilation.txt

@cFire can you give the sorted text link for download?

@cFire can you please share the link to the sorted one please?

that would be great yes

There is no sorted text link. You have to download the magnet link provided above and run the line cFire provided.

Hello there!
Here is my experience of importing Breach Compilation in Postgres:
https://gist.github.com/spacepatcher/8f94289236612a40ed93efc5645c0ccd

If anybody wants to quickly look up their own address and is unable to set it up themselves, I've set up a telegram bot for that.
It's running on a weak server, so please don't look up more than a few addresses. And I might remove that functionality any time. but if you have no better option whatsoever, it's something ;)

Direct Telegram Link
Usage: /q my@email.com

I also feature regex searching the results.

Does anyone know where is this new leak? https://www.hackread.com/3000-databases-200-million-unique-accounts-exposed-dark-web/
with email and password as the link @modInfo provided.

@adon90 if you happen to find out, please let us know.

But I assume that is a database that was sold in some darkweb marketplace, not a public dump.

is it also possible to use a domain name instead of a full email adres, like *@outlook.com

can anyone suggest me how to import all the db in sql?

Can you explain me why you have used 'ohP '(?<=:).*$’ ' in grep command? Thanks

certain lines have; as delimiter and not :
How to extract the passwords in such a case
Like in file data/n/a/0 line 955

@mayankmetha replace all ; with :

How can I specifically see one of the dumps, e.g. only the LinkedIn one or another one ?

can someone zip and repost breachcompilation_sorted.txt?

Ran sort/uniq the breachcompilation list and repackaged it. Compressed size runs around a gig.
magnet:?xt=urn:btih:ac0df29b6ae2061a1f547056eee37343646e11de&dn=breaches.7z&tr=http%3A%2F%2Ftracker1.wasabii.com.tw%3A6969%2Fannounce&tr=udp%3A%2F%2Fexplodie.org%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2Fipv4.tracker.harry.lu%3A80%2Fannounce

full base

magnet:?xt=urn:btih:7ffbcd8cee06aba2ce6561688cf68ce2addca0a3&dn=BreachCompilation&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80&tr=udp%3A%2F%2Ftracker.leechers-paradise.org%3A6969&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969&tr=udp%3A%2F%2Fglotorrents.pw%3A6969&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337

is there any way we can edit the query.sh file and make it search like *@gmail.com and get all gmails. ?

Are there any special characters in password list? Did you delete a partial of password? I just wondered that I can check whether or not my client is vulnerable.

For anyone who has DHT disabled and need trackers, I would recommend @ngosang @ https://github.com/ngosang/trackerslist