Reply
 
LinkBack Thread Tools Display Modes
  #1   Report Post  
Old February 2nd 07, 03:03 PM posted to rec.games.chess.computer
external usenet poster
 
First recorded activity by ChessBanter: Aug 2006
Posts: 58
Default unix tool for removing duplicates from a large .pgn file?

Hi,

I'm collecting as much chess matches played (by humans) as possible.
Now I have around 400MB of pgn-files and I would like to remove the
inevitable duplicates.
Does anyone know a tool for unix (linux/macos x/bsd/aix/etc.) doing so?

--
--------------------------------------------------------------------
Phone: +31-6-41278122, PGP-key: 1F28D8AE, www.vanheusden.com


  #2   Report Post  
Old February 2nd 07, 06:49 PM posted to rec.games.chess.computer
external usenet poster
 
First recorded activity by ChessBanter: Sep 2003
Posts: 41
Default unix tool for removing duplicates from a large .pgn file?

Folkert van Heusden wrote:
Hi,

I'm collecting as much chess matches played (by humans) as possible.
Now I have around 400MB of pgn-files and I would like to remove the
inevitable duplicates.
Does anyone know a tool for unix (linux/macos x/bsd/aix/etc.) doing so?


pgn-extract

or

scid

--
GCP
  #3   Report Post  
Old February 2nd 07, 09:26 PM posted to rec.games.chess.computer
external usenet poster
 
First recorded activity by ChessBanter: Feb 2007
Posts: 2
Default unix tool for removing duplicates from a large .pgn file?

Folkert van Heusden wrote:
Hi,

I'm collecting as much chess matches played (by humans) as possible.
Now I have around 400MB of pgn-files and I would like to remove the
inevitable duplicates.
Does anyone know a tool for unix (linux/macos x/bsd/aix/etc.) doing so?

png-extract

http://www.cs.kent.ac.uk/people/staff/djb/pgn-extract/

will do it for you. It has no GUI, will do what you want.

ChessDB (based on Scid, but developed further), will do too.

http://chessdb.sourceforge.net/

it has a GUI interface, so is quite different in use. Depends what you
want.

--
Dave (from the UK)

Please note my email address changes periodically to avoid spam.
It is always of the form:
Hitting reply will work for a few months only - later set it manually.

http://chessdb.sourceforge.net/ - a Free open-source Chess Database
  #4   Report Post  
Old February 2nd 07, 10:42 PM posted to rec.games.chess.computer
external usenet poster
 
First recorded activity by ChessBanter: Aug 2006
Posts: 58
Default unix tool for removing duplicates from a large .pgn file?

Gian-Carlo/Dave: thanks! pgn-extract does exactly what I was looking for!


Reply
Thread Tools
Display Modes

Posting Rules

Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On



All times are GMT +1. The time now is 11:54 AM.

Powered by vBulletin® Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Copyright 2004-2019 ChessBanter.
The comments are property of their posters.
 

About Us

"It's about Chess"

 

Copyright © 2017