Skip to content

Commit

Permalink
Fix for Uint8 / Uint16 Unicode string (#220)
Browse files Browse the repository at this point in the history
* Fix truncated string when underlying database engine do not use Uint8 unicode data format.

* Update for full unicode specification.

Usage of field size is deleted, only data size remain to determine the
length of data we need to read.

This fix the bug that cause the driver to truncate string if the above
two conditions are verified :

- the underlying database engine use Uint16 in place of Uint8 for
encoding unicode string.
- the readed string size in greater than the half of the field size.

Example for Uint16 encoding :
- field size 10, string size 4 -> no bug
- field size 10, string size 6 -> bug, only the firsts fives characters
are returned

According to MS Doc The function SQLDescribeCol, return  among other
things, the size in characters, not byte, of the field.

the other ODBC functions used to retrieve data use length indicator
in byte. (SQLBindCol, SQLFetchScroll)

Another bug may be corrected when using Uint8 and many 3-bytes
unicode characters. ! TO BE TESTED !

* Update test case for unicode char

* Update for correct offset calculation

* Delete test.jl

Delete personal test file

* add @inbound
  • Loading branch information
pascalr0410 authored and quinnj committed May 7, 2019
1 parent a0cc3e5 commit d322258
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions src/Query.jl
Original file line number Diff line number Diff line change
Expand Up @@ -207,9 +207,9 @@ function cast!(::Type{Union{String, Missing}}, source, col)
elsize = source.sizes[col] + 1
inds = source.indcols[col]
@inbounds for i in 1:len
ind = min(inds[i]|>Int, elsize|>Int)
ind = inds[i]
length = bytes2codeunits(T, max(ind, 0))
c[i] = ind == API.SQL_NULL_DATA ? missing : (length == 0 ? "" : String(transcode(UInt8, data[cur:(cur + length - 1)])))
c[i] = ind == API.SQL_NULL_DATA ? missing : (length == 0 ? "" : transcode(String, data[cur:(cur + length - 1)]))
cur += elsize
end
return c
Expand Down

0 comments on commit d322258

Please sign in to comment.