Changelog History
Page 1
-
v1.0.7 Changes
March 06, 2022- ๐ fix: :doc:
/scripts/csvcut
extracts the correct columns when :code:--line-numbers
is set. - ๐ fix: Restore Python 2.7 support in edge cases.
- ๐ feat: Use 1024 byte sniff-limit by default across csvkit. Improve csvstat performance up to 10x.
- ๐ feat: Add support for
.xz
(LZMA) compressed input files. - โ Add Python 3.10 support.
- โฌ๏ธ Drop Python 3.5 support (end-of-life was September 30, 2020).
- ๐ fix: :doc:
-
v1.0.6 Changes
July 13, 2021๐ Changes:
- ๐จ :doc:
/scripts/csvstat
no longer prints "Row count: " when :code:--count
is set. - :doc:
/scripts/csvclean
, :doc:/scripts/csvcut
, :doc:/scripts/csvgrep
no longer error if standard input is null.
๐ Fixes:
- 0๏ธโฃ :doc:
/scripts/csvformat
creates default headers when :code:--no-header-row
is set, as documented. - :doc:
/scripts/csvstack
no longer errors when :code:--no-header-row
is combined with :code:--groups
or :code:--filenames
.
- ๐จ :doc:
-
v1.0.5 Changes
March 02, 2020๐ Changes:
- โฌ๏ธ Dropped Python 3.4 support (end-of-life was March 18, 2019).
๐ Improvements:
- Output error message for memory error even if not :code:
--verbose
.
๐ Fixes:
- ๐ Fix regression in 1.0.4, which caused numbers like
4.5
to be parsed as dates. - :doc:
/scripts/in2csv
Fix error reporting if :code:--names
used with non-Excel file.
-
v1.0.4 Changes
March 16, 2019๐ Changes:
- โฌ๏ธ Dropped Python 3.3 support (end-of-life was September 29, 2017).
๐ Improvements:
- :doc:
/scripts/csvsql
adds a :code:--chunk-size
option to set the chunk size when batch inserting into a table. - โ csvkit is now tested against Python 3.7.
๐ Fixes:
- :code:
--names
works with :code:--skip-lines
. - ๐ Dates and datetimes without punctuation can be parsed with :code:
--date-format
and :code:datetime-format
. - Error messages about column indices use 1-based numbering unless :code:
--zero
is set. - :doc:
/scripts/csvcut
no longer errors on :code:--delete-empty-rows
with short rows. - :doc:
/scripts/csvjoin
no longer errors if given a single file. - โก๏ธ :doc:
/scripts/csvsql
supports UPDATE commands. - :doc:
/scripts/csvstat
no longer errors on non-finite numbers. - :doc:
/scripts/csvstat
respects all command-line arguments when :code:--count
is set. - :doc:
/scripts/in2csv
CSV-to-CSV conversion respects :code:--linenumbers
when buffering. - :doc:
/scripts/in2csv
writes XLS sheets without encoding errors in Python 2.
-
v1.0.3 Changes
March 11, 2018๐ Improvements:
- :doc:
/scripts/csvgrep
adds a :code:--any-match
(:code:-a
) flag to select rows where any column matches instead of all columns. - :doc:
/scripts/csvjson
no longer emits a property if its value is null. - :doc:
/scripts/csvjson
adds :code:--type
and :code:--geometry
options to emit non-Point GeoJSON features. - :doc:
/scripts/csvjson
adds a :code:--no-bbox
option to disable the calculation of a bounding box. - ๐ :doc:
/scripts/csvjson
supports :code:--stream
for newline-delimited GeoJSON. - :doc:
/scripts/csvsql
adds a :code:--unique-constraint
option to list names of columns to include in a UNIQUE constraint. - :doc:
/scripts/csvsql
adds :code:--before-insert
and :code:--after-insert
options to run commands before and after the INSERT command. - :doc:
/scripts/csvpy
reports an error message if input is provided via STDIN. - :doc:
/scripts/in2csv
adds a :code:--encoding-xls
option to specify the encoding of the input XLS file. - ๐ :doc:
/scripts/in2csv
supports :code:--no-header-row
on XLS and XLSX files. - โ Suppress agate warning about column names not specified when using :code:
--no-header-row
. - Prompt the user if additional input is expected (i.e. if no input file or piped data is provided).
- โ
Update to
agate-excel 0.2.2 <http://agate-excel.readthedocs.io/en/latest/#changelog>
,agate-sql 0.5.3 <http://agate-sql.readthedocs.io/en/latest/#changelog>
.
๐ Fixes:
- :doc:
/scripts/csvgrep
accepts utf-8 arguments to the :code:--match
and :code:--regex
options in Python 2. - :doc:
/scripts/csvjson
streams input and output only if :code:--snifflimit
is :code:0
. - :doc:
/scripts/csvsql
sets a DECIMAL's precision and scale and a VARCHAR's length to avoid dialect-specific errors. - :doc:
/scripts/csvstack
no longer opens all files at once. - :doc:
/scripts/in2csv
respects :code:--no-header-row
when :code:--no-inference
is set. - :doc:
/scripts/in2csv
CSV-to-CSV conversion streams input and output only if :code:--snifflimit
is :code:0
. - ๐ :doc:
/scripts/in2csv
supports GeoJSON files with:geometry
set tonull
, missing Pointcoordinates
, altitude coordinate values.
โ csvkit is no longer tested on PyPy.
- :doc:
-
v1.0.2 Changes
April 28, 2017๐ Improvements:
- โ Add a :code:
--version
flag. - โ Add a :code:
--skip-lines
option to skip initial lines (e.g. comments, copyright notices, empty rows). - โ Add a :code:
--locale
option to set the locale of any formatted numbers. - โ Add a :code:
--date-format
option to set a strptime date format string. - โ Add a :code:
--datetime-format
option to set a strptime datetime format string. - ๐ Make :code:
--blanks
a common argument across all tools. - :code:
-I
is the short option for :code:--no-inference
. - ๐ :doc:
/scripts/csvclean
, :doc:/scripts/csvformat
, :doc:/scripts/csvjson
, :doc:/scripts/csvpy
support :code:--no-header-row
. - :doc:
/scripts/csvclean
is faster and no longer requires exponential time in the worst case. - ๐ :doc:
/scripts/csvformat
supports :code:--linenumbers
and--zero
(no-op). - ๐ :doc:
/scripts/csvjoin
supports :code:--snifflimit
and :code:--no-inference
. - ๐ :doc:
/scripts/csvpy
supports :code:--linenumbers
(no-op) and :code:--zero
(no-op). - :doc:
/scripts/csvsql
adds a :code:--prefix
option to add expressions like OR IGNORE or OR REPLACE following the INSERT keyword. - :doc:
/scripts/csvsql
adds a :code:--overwrite
flag to drop any existing table with the same name before creating. - :doc:
/scripts/csvsql
accepts a file name for the :code:--query
option. - ๐ :doc:
/scripts/csvsql
supports :code:--linenumbers
(no-op). - :doc:
/scripts/csvsql
adds a :code:--create-if-not-exists
flag to not abort if the table already exists. - :doc:
/scripts/csvstat
adds a :code:--freq-count
option to set the maximum number of frequent values to display. - ๐ :doc:
/scripts/csvstat
supports :code:--linenumbers
(no-op). - ๐จ :doc:
/scripts/in2csv
adds a :code:--names
flag to print Excel sheet names. - :doc:
/scripts/in2csv
adds a :code:--write-sheets
option to write the named Excel sheets to files. - :doc:
/scripts/sql2csv
adds an :code:--encoding
option to specify the encoding of the input query file.
๐ Fixes:
- :doc:
/scripts/csvgrep
no longer ignores common arguments if :code:--linenumbers
is set. - ๐ :doc:
/scripts/csvjson
supports Decimal. - ๐ :doc:
/scripts/csvpy
again supports IPython. - โช :doc:
/scripts/csvsql
restores support for :code:--no-constraints
and :code:--db-schema
. - :doc:
/scripts/csvstat
will no longer crash when :code:--freq
is set. - โช :doc:
/scripts/in2csv
restores support for :code:--no-inference
for Excel files. - โช :doc:
/scripts/in2csv
restores support for converting Excel files from standard input. - :doc:
/scripts/in2csv
accepts utf-8 arguments to the :code:--sheet
option in Python 2.
- โ Add a :code:
-
v1.0.1 Changes
December 29, 2016๐ This is a minor release which fixes several bugs reported in the :code:
1.0.0
release earlier this week. It also significantly improves the output of :doc:/scripts/csvstat
and adds a :code:--csv
output option to that command.- :doc:
/scripts/csvstat
will no longer crash when a :code:Number
column has :code:None
as a frequent value. (#738) - ๐ :doc:
/scripts/csvlook
docs now note that output tables are Markdown-compatible. (#734) - ๐ :doc:
/scripts/csvstat
now supports a :code:--csv
flag for tabular output. (#584) - :doc:
/scripts/csvstat
output is now easier to read. (#714) - ๐ :doc:
/scripts/csvpy
now has a better description when using the :code:--agate
flag. (#729) - ๐ Fix a Python 2.6 bug preventing :doc:
/scripts/csvjson
from parsing utf-8 files. (#732) - โก๏ธ Update required version of unittest to latest. (#727)
- :doc:
-
v1.0.0 Changes
December 27, 2016๐ This is the first major release of csvkit in a very long time. The entire backend has been rewritten to leverage the
agate <http://agate.rtfd.io>
_ data analysis library, which was itself inspired by csvkit. The new backend provides better type detection accuracy, as well as some new features.๐ Because of the long and complex cycle behind this release, the list of changes should not be considered exhaustive. In particular, the output format of some tools may have changed in small ways. Any existing data pipelines using csvkit should be tested as part of the upgrade.
๐ Much of the credit for this release goes to
James McKinney <https://github.com/jpmckinney>
_, who has almost single-handedly kept the csvkit fire burning for a year. Thanks, James!Backwards-incompatible changes:
- ๐ :doc:
/scripts/csvjoin
now renames duplicate columns with integer suffixes to prevent collisions in output. - :doc:
/scripts/csvsql
now generatesDateTime
columns instead ofTime
columns. - :doc:
/scripts/csvsql
now generatesDecimal
columns instead ofInteger
,BigInteger
, andFloat
columns. - :doc:
/scripts/csvsql
no longer generates max-length constraints for text columns. - The
--doublequote
long flag is gone, and the-b
short flag is now an alias for--no-doublequote
. - When using the
--columns
or--not-columns
options, you must not have spaces around the comma-separated values, unless the column names contain spaces. - When sorting, null values are now greater than other values instead of less than.
- ๐
CSVKitReader
,CSVKitWriter
,CSVKitDictReader
, andCSVKitDictWriter
have been removed. Useagate.csv.reader
,agate.csv.writer
,agate.csv.DictReader
andagate.csv.DictWriter
. - โฌ๏ธ Drop Python 2.6 support (end-of-life was October 29, 2013).
- โฌ๏ธ Drop support for older versions of PyPy.
- If
--no-header-row
is set, the output will have column namesa
,b
,c
, etc. instead ofcolumn1
,column2
,column3
, etc. - csvlook renders a simpler, markdown-compatible table.
๐ Improvements:
- โ csvkit is now tested against Python 3.6. (#702)
import csvkit as csv
will now defer to agate readers/writers.- ๐ :doc:
/scripts/csvgrep
supports--no-header-row
. - ๐ :doc:
/scripts/csvjoin
supports--no-header-row
. - :doc:
/scripts/csvjson
streams input and output if the--stream
and--no-inference
flags are set. - ๐ :doc:
/scripts/csvjson
supports--snifflimit
and--no-inference
. - :doc:
/scripts/csvlook
adds--max-rows
,--max-columns
and--max-column-width
options. - ๐ :doc:
/scripts/csvlook
supports--snifflimit
and--no-inference
. - ๐ :doc:
/scripts/csvpy
supports--agate
to read a CSV file into an agate table. - โ
csvsql
supports customSQLAlchemy dialects <http://docs.sqlalchemy.org/en/latest/dialects/>
_. - ๐ :doc:
/scripts/csvstat
supports--names
. - :doc:
/scripts/in2csv
CSV-to-CSV conversion streams input and output if the--no-inference
flag is set. - :doc:
/scripts/in2csv
CSV-to-CSV conversion usesagate.Table
. - :doc:
/scripts/in2csv
GeoJSON conversion adds columns for geometry type, longitude and latitude. - ๐ Documentation: Update tool usage, remove shell prompts, document connection string, correct typos.
๐ Fixes:
- ๐ Fixed numerous instances of open files not being closed before utilities exit.
- ๐ Change
-b
,--doublequote
to--no-doublequote
, as doublequote is True by default. - :doc:
/scripts/in2csv
DBF conversion works with Python 3. - :doc:
/scripts/in2csv
correctly guesses format when file has an uppercase extension. - :doc:
/scripts/in2csv
correctly interprets--no-inference
. - ๐ :doc:
/scripts/in2csv
again supports nested JSON objects (fixes regression). - ๐จ :doc:
/scripts/in2csv
with--format geojson
will print a JSON object instead ofOrderedDict([(...)])
. - ๐ :doc:
/scripts/csvclean
with standard input works on Windows. - :doc:
/scripts/csvgrep
returns the input file's line numbers if the--linenumbers
flag is set. - :doc:
/scripts/csvgrep
can match multiline values. - :doc:
/scripts/csvgrep
correctly operates on ragged rows. - :doc:
/scripts/csvsql
correctly escapes%
` characters in SQL queries. - :doc:
/scripts/csvsql
adds standard input only if explicitly requested. - ๐ :doc:
/scripts/csvstack
supports stacking a single file. - :doc:
/scripts/csvstat
always reports frequencies. - The
any_match
argument ofFilteringCSVReader
now works correctly. - All tools handle empty files without error.
- ๐ :doc:
-
v0.9.2
September 04, 2018 -
v0.9.1 Changes
March 31, 2015- โ Add Antonio Lima to AUTHORS.
- โ Add support for ndjson. (#329)
- โ Add missing docs for csvcut -C. (#227)
- ๐ Reorganize docs so TOC works better. (#339)
- ๐ Render docs locally with RTD theme.
- ๐ Fix header in "tricks" docs.
- โ Add install instructions to tutorial. (#331)
- โ Add killer examples to doc index. (#328)
- Reorganize doc index
- ๐ Fix broken csvkit module documentation. (#327)
- ๐ Fix version of openpyxl to work around encoding issue. (#391, #288)