Remove Duplicate Rows (CSV/Excel)
Find and remove repeated rows by the columns that matter, with a file of what was removed.
Or paste the data (from Excel, Google Sheets or a CSV)
What counts as a duplicate
Result
Largest groups
| Compared values | Kept row | Removed rows |
|---|
The removed-duplicates file has two extra columns: Duplicate of row (the row that was kept, numbered as in the original file, header = row 1) and Copies in group.
About the Remove Duplicate Rows (CSV/Excel)
Remove the repeated rows from a CSV, TSV or Excel file — whole-row duplicates, or rows that repeat a key such as an email address, phone number, customer ID, or a combination such as name and date of birth. Values are compared after the clean-ups you choose: spaces at the ends, upper and lower case, repeated spaces, accents (é = e) and, for phone numbers and IDs, all spaces and punctuation. Rows whose key is empty are not treated as duplicates of each other, so records without an email are not thrown away.
Of each group of duplicates you keep the first row, the last one (the latest entry in a log) or the most complete one (the row with the most filled-in cells). You get the cleaned file in its original order and a separate file of the removed rows, each with the row it duplicated and how many copies its group had — so nothing disappears without a trace.
How to use it
- Choose a CSV, TSV or Excel file, or paste cells copied from a spreadsheet.
- Check First row is the header and, for a workbook, the sheet.
- Under Rows are duplicates when they match in, keep Every column or choose Chosen columns and tick the key columns.
- Choose which differences to ignore and which row of each group to keep.
- Look at the result and the largest groups, then download the cleaned file and, if you want a record, the removed duplicates.
Examples
Name,Email Asha Rao,[email protected] ASHA RAO, [email protected] Ravi,[email protected]
Kept: Asha Rao (row 2) and Ravi (row 4). Removed: row 3, “Duplicate of row” 2, “Copies in group” 2.
With spaces at the ends and case ignored, the two addresses are the same. Compare only the Email column, so different spellings of the name do not matter.
98765 43210 9876543210 98765-43210
One row kept
Tick All spaces and punctuation. A number written with the country code (+91 98765 43210) still differs, because the digits are not the same.
Ravi, [email protected], (no phone) Ravi Kumar, [email protected], 91234 56780
The second row is kept because it has more filled-in cells.
Common uses
- Cleaning a mailing or WhatsApp list before a campaign so nobody gets the same message twice.
- Removing repeated orders, invoices or transactions from exports that overlap.
- Merging contact lists from several sources and keeping one row per person.
- Keeping the latest status per ID from a log (keep the last row).
How rows are compared
For every row, the chosen columns are cleaned up with the options you ticked and joined into a key; rows with the same key form a group, and one row of each group is kept. Comparing uses the text as it appears — in a workbook, the value as Excel displays it, as Excel’s own Remove Duplicates does — so 1,250.00 and 1250 count as different. The header row is never compared.
- Spaces at the start and end:
" Asha "="Asha". - Upper and lower case:
[email protected]=[email protected]. - Repeated spaces:
Asha Rao=Asha Rao(tabs and line breaks too). - Accents:
José=Jose,Müller=Muller,Straße=Strasse. Only the accents of Latin, Greek and Cyrillic letters are ignored: the vowel signs of Hindi, Tamil and other Indian scripts are part of the word, soकलandकालstay different. - All spaces and punctuation:
98765-43210=98765 43210=9876543210.
What you download
The cleaned file keeps the header and the kept rows in their original order, in CSV, TSV or Excel. From a workbook, Excel output keeps the cells’ numbers, dates and formats; from a CSV, whole numeric columns become numbers while codes such as 007 stay text. The removed-duplicates file has the same columns plus Duplicate of row — the kept row’s number in the original file, counting the header as row 1, as a spreadsheet does — and Copies in group, the number of rows that shared the key.
Limitations
- CSV files up to 100 MB and workbooks up to 50 MB, with at most 10 million cells; one sheet at a time.
- Formulas in workbooks are read as their last saved values; the downloaded file contains values, not formulas, and no cell styles other than number formats.
- Near-duplicates with typing differences (Asha Rao / Aasha Rao) are not matched — only values that are equal after the chosen clean-ups.
- Phone numbers with and without a country code are different values; remove the code first (for example with Find & replace in the CSV Editor) to match them.
Privacy
Everything happens in your browser. What you enter or open here is not uploaded or stored by MySmartCoPilot.
Frequently asked questions
How do I remove duplicates based on one column?
Choose Chosen columns and tick only that column, for example Email. Rows with the same email are duplicates even if the other columns differ; the row you choose to keep stays.
Which duplicate is kept?
The first one by default, as in Excel. Choose The last to keep the most recent entry of a log, or The most complete to keep the row with the most filled-in cells.
Are rows with an empty email removed as duplicates of each other?
No — not while Never treat rows as duplicates when all compared cells are empty is ticked. Untick it if empty keys should count as the same value.
Can I see what was removed?
Yes. The Largest groups table shows the kept and removed row numbers, and Download the removed duplicates gives every removed row with the row it duplicated and the size of its group.
Does it keep the order of my rows?
Yes. The kept rows stay in their original order; nothing is sorted.
Is my file uploaded?
No. The file is read and processed in your browser, and the results are created on your device.