Valloric / ycmd / 1857 / 1
94%
master: 94%

DEFAULT BRANCH: master
Ran
Files 40
Run time 1s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 84.723% (+0.2%) from 84.569%
USE_CLANG_COMPLETER=true YCMD_PYTHON_VERSION=2.7 COVERAGE=true

push

travis-ci

homu 
Auto merge of #455 - puremourning:unicode-investigation, r=Valloric

[READY] Fix issues with multi-byte characters

## Summary

This change introduces more general support for non-ASCII characters in buffers handled by YCMD.

In ycmd's public API, all offsets are byte offsets into the UTF-8 encoded buffers. We also assume (because, we have no other choice) that files stored on disk are also UTF-8 encoded. Internally, almost all of ycmd's functionality operates on unicode strings (python 2 `unicode()` and python 3 `str()` objects, transparently via `future`). Many of the downstream completion engines expect unicode code points as the offsets in their APIs. One special case is the `ycm_core` library (identifier completer and clang completer), which requires instances of the _native_ `str` type. All strings used within the c++ using `boost::python` require passing through `ToCppStringCompatible`

Previously, we were largely just assuming that `code point == byte offset` - i.e. all buffers contained only ASCII characters. This worked up to a point, but more by luck than judgement in a number of places.

## References

In combination with a YCM change and PR https://github.com/Valloric/ycmd/pull/453, I hope this:

- fixes https://github.com/Valloric/ycmd/issues/109
- fixes https://github.com/Valloric/YouCompleteMe/issues/2096
- fixes https://github.com/Valloric/YouCompleteMe/issues/2088
- fixes https://github.com/Valloric/YouCompleteMe/issues/2069
- fixes https://github.com/Valloric/YouCompleteMe/issues/2066
- fixes https://github.com/Valloric/YouCompleteMe/issues/1378

## Overview of changes

The changes fall into the following areas:

- Providing access to and conversion to/from code points and byte offsets (`request_wrap.py`)
- Changing certain algorithms/features to work entirely in codepoint space when they are trying to operate on logical 'characters' within the buffer (see known issues for why this isn't perfect, but probably most of the way there)
... (continued)

3028 of 3574 relevant lines covered (84.72%)

0.85 hits per line

Source Files on job 1857.1 (USE_CLANG_COMPLETER=true YCMD_PYTHON_VERSION=2.7 COVERAGE=true)