ChiliProject is not maintained anymore. Please be advised that there will be no more updates.
We do not recommend that you setup new ChiliProject instances and we urge all existing users to migrate their data to a maintained system, e.g. Redmine. We will provide a migration script later. In the meantime, you can use the instructions by Christian Daehn.
Repositories encodings are broken (Bug #433)
Description
Associated revisions
Propagates time tracking to the parent project (closes #433). Time report enhancements.
git-svn-id: http://redmine.rubyforge.org/svn/trunk@1176 e93f8b46-1217-0410-a6f0-8f06a7374b81
History
Updated by Eric Davis at 2011-05-29 06:01 pm
Please provide a description about the bug when posting it.
- Priority changed from High to Normal
- Category set to SCM
- Status changed from Open to Needs more information
Updated by Toshi MARUYAMA at 2011-05-30 02:57 am
- Status changed from Needs more information to Open
Updated by Toshi MARUYAMA at 2011-05-30 03:46 am
$ python >>> s = "japanese" >>> u = s.decode("utf-8").encode("utf-16") >>> u '\xff\xfej\x00a\x00p\x00a\x00n\x00e\x00s\x00e\x00'
Updated by Toshi MARUYAMA at 2011-05-30 03:57 am
$ python >>> s = "日本語" >>> u = s.decode("utf-8").encode("utf-16") >>> u '\xff\xfe\xe5e,g\x9e\x8a'
Updated by Toshi MARUYAMA at 2011-05-30 04:01 am
to_utf8_for_attachments() at source:app/helpers/attachments_helper.rb@13de4cc4#L31 returns "e,g" in "日本語" of UTF-16.
Updated by Toshi MARUYAMA at 2011-05-30 04:07 am
to_utf8() at source:app/helpers/repositories_helper.rb@2e1bcb2a#L119 converts UTF-16 to UTF-8.
Updated by Toshi MARUYAMA at 2011-06-03 12:11 am
$ irb ruby-1.8.7-p330 :001 > require 'iconv' => true ruby-1.8.7-p330 :002 > Iconv.conv("UTF-16","UTF-8","日本語") => "\377\376\345e,g\236\212" ruby-1.8.7-p330 :003 > Iconv.conv("UTF-16","UTF-8","japanese") => "\377\376j\000a\000p\000a\000n\000e\000s\000e\000"
Updated by Eric Davis at 2011-06-03 06:06 pm
Fixed in my ticket/release-v2.0.0/444-scm-encoding branch and merged into release-v2.0.0 in 3b64c60.
- Target version set to 2.0.0
- Assignee set to Eric Davis
- Status changed from Open to Closed
Updated by Toshi MARUYAMA at 2011-06-03 10:29 pm
Not fixed.
Why you don't backout 9baa310486d782b ?
Do you understand 'UTF-16'?
- Status changed from Closed to Open
Updated by Toshi MARUYAMA at 2011-06-03 10:33 pm
Did you run and confirm filesystem test repository?
Updated by Eric Davis at 2011-06-03 10:57 pm
Toshi MARUYAMA wrote:
Not fixed.
Did you run and confirm filesystem test repository?
1. It has been confirmed fixed according to my own tests and also confirmed on the CI server
Why you don't backout 9baa310486d782b ?
2. 9baa310486d782b has nothing to do with this. Both assert_tag and assert_select are testing the same thing.
Do you understand 'UTF-16'?
3. Yes
Status changed from Closed to Open
4. Please do not reopen issues without contributing any more information. This is the second time today I've had to say this today. https://www.chiliproject.org/issues/440#note-21
- Status changed from Open to Closed
Updated by Toshi MARUYAMA at 2011-06-03 11:06 pm
Description images are on ChiliProject.
It is easy to reproduce.
UTF-16 has '\0', so it is easy to understand ChiliProject has 'MOJI-BAKE' (broken encoding).
Before you changed filesytem functional test, output has '\0'.
So, it is easy to understand why test fails.
Your modification filesystem test is wrong.
- Status changed from Closed to Open
Updated by Toshi MARUYAMA at 2011-06-03 11:11 pm
Your recent revisions are about ChangeSets.
Filesystem repository does not have ChangeSets.
Filesystem repository reads file directly for equivalent "svn/hg/git cat".
Do you really understand character encoding and internal ChiliProject/Redmine?
Updated by Toshi MARUYAMA at 2011-06-03 11:15 pm
This includes \000j\000a\000p\000a\000n\000e\000s\000e\000\r\000\n
Updated by Toshi MARUYAMA at 2011-06-03 11:36 pm
I changed this lines as following.
source:test/functional/repositories_filesystem_controller_test.rb@a428fa60#L81
japanese -> english
def test_show_utf16 with_settings :repositories_encodings => 'UTF-16' do get :entry, :id => PRJ_ID, :path => ['japanese', 'utf-16.txt'] assert_response :success assert_select "tr" do assert_select "th.line-num" do assert_select "a", :text => /2/ end assert_select "td", :content => /english/ end end
But, test passes.
Your test is wrong.
You don't understand 'UTF-16'.
Updated by Toshi MARUYAMA at 2011-06-04 01:18 am
Eric Davis wrote:
4. Please do not reopen issues without contributing any more information. This is the second time today I've had to say this today. https://www.chiliproject.org/issues/440#note-21
I give you more information in English.
You don't understand my information.
Updated by Toshi MARUYAMA at 2011-06-04 04:57 am
Eric Davis wrote:
Do you understand 'UTF-16'?
3. Yes
You don't understand UTF-16.
$ rake test:scm:setup:filesystem ./filesystem_repository/ ./filesystem_repository/test ./filesystem_repository/japanese/ ./filesystem_repository/japanese/utf-16.txt ./filesystem_repository/japanese/euc-jp.txt ./filesystem_repository/japanese/big-file.txt ./filesystem_repository/dir/ ./filesystem_repository/dir/dirfile ./filesystem_repository/dir/subdir/ $ export LANG=en_US.UTF-8 $ locale LANG=en_US.UTF-8 LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_PAPER="en_US.UTF-8" LC_NAME="en_US.UTF-8" LC_ADDRESS="en_US.UTF-8" LC_TELEPHONE="en_US.UTF-8" LC_MEASUREMENT="en_US.UTF-8" LC_IDENTIFICATION="en_US.UTF-8" LC_ALL= $ cat tmp/test/filesystem_repository/japanese/utf-16.txt \ | iconv -f utf-16 -t utf-8 | cat -n 1 UTF-16 2 japanese 3 日本語 4 ã«ã»ã‚“ã” 5 ニホンゴ 6 日本語 7 ã«ã»ã‚“ã” 8 ニホンゴ 9 日本語 10 ã«ã»ã‚“ã” 11 ニホンゴ 12 $ cat tmp/test/filesystem_repository/japanese/utf-16.txt | od -a 0000000 del ~ U nul T nul F nul - nul 1 nul 6 nul cr nul 0000020 nl nul j nul a nul p nul a nul n nul e nul s nul 0000040 e nul cr nul nl nul e e , g rs nl cr nul nl nul 0000060 k 0 { 0 dc3 0 T 0 cr nul nl nul K 0 [ 0 0000100 s 0 4 0 cr nul nl nul e e , g rs nl cr nul 0000120 nl nul k 0 { 0 dc3 0 T 0 cr nul nl nul K 0 0000140 [ 0 s 0 4 0 cr nul nl nul e e , g rs nl 0000160 cr nul nl nul k 0 { 0 dc3 0 T 0 cr nul nl nul 0000200 K 0 [ 0 s 0 4 0 cr nul nl nul cr nul nl nul 0000220
Updated by Toshi MARUYAMA at 2011-06-04 05:34 am
Eric Davis wrote:
2. 9baa310486d782b has nothing to do with this. Both assert_tag and assert_select are testing the same thing.
Your test is wrong.
This test passes.
def test_show_eucjp with_settings :repositories_encodings => 'UTF-8' do get :entry, :id => PRJ_ID, :path => ['japanese', 'euc-jp.txt'] assert_response :success assert_select "td", {:count=>1, :text=>"japanese"} end end
But, this test fails.
def test_show_utf16 with_settings :repositories_encodings => 'UTF-16' do get :entry, :id => PRJ_ID, :path => ['japanese', 'utf-16.txt'] assert_response :success assert_select "td", {:count=>1, :text=>"japanese"} end end
1) Failure: test_show_utf16(RepositoriesFilesystemControllerTest) [test/functional/repositories_filesystem_controller_test.rb:93:in `test_show_utf16' test/test_helper.rb:89:in `with_settings' test/functional/repositories_filesystem_controller_test.rb:90:in `test_show_utf16']: <"japanese"> expected but was <"U\000T\000F\000-\0001\0006\000\r\000">. <false> is not true.
UTF-16 is not compatible with ASCII.
EUC-JP and UTF-8 are compatible with ASCII.
You don't understand UTF-16.
Updated by Eric Davis at 2011-06-05 05:20 pm
- Target version deleted (
2.0.0) - Assignee deleted (
Eric Davis)
Updated by deleted deleted at 2011-06-05 07:30 pm
@Toshi: It's quite over the top imho to (re-)open THREE times a ticket despite having been asked not to do so, resp. having been asked for more information (which 'You should backout 9baa3104.' certainly is not). Please try not to clutter issues with updates and finally, writing "Your test is wrong. You don't understand 'UTF-16'." is very unfriendly and really not helpful. - thank you
Updated by Toshi MARUYAMA at 2011-06-06 12:52 am
Hans-Peter Suter wrote:
Please try not to clutter issues with updates and finally, writing "Your test is wrong. You don't understand 'UTF-16'." is very unfriendly and really not helpful. - thank you
It is easy to know why RepositoriesFilesystemControllerTest.test_show_utf16 fails.
regression