CONVERTCP Activity
Text File Codepage Converter for the Windows command line
Brought to you by:
german-one
Activity for CONVERTCP
-
CONVERTCP released /readme.md
-
-
-
-
Steffen modified a comment on a wiki page
About Virtual Terminal processing for version 6.3 and newer Most of the current command line utilities don't support Virtual Terminal processing yet. In this case ANSI escape sequences are not used to control the console output and their textual expressions get printed to the screen. Even if I expect that VT processing will be only barely used along with CONVERTCP, it won't hurt to enable it once that Windows 10 provides this possibility. Example using CONVERTCP v. 6.3: >nul chcp 65001 echo +ABsAWw-93;42m+JYgliCWIJZMlkyWTJZIlkiWSJZElkSWR-...
-
About Virtual Terminal processing for version 6.3 and newer Most of the current command line utilities don't support Virtual Terminal processing yet. In this case ANSI escape sequences are not used to control the console output and their textual expressions get printed to the screen. Even if I expect that VT processing will be only barely used along with CONVERTCP, it won't hurt to enable it once that Windows 10 provides this possibility. Example using CONVERTCP v. 6.3: >nul chcp 65001 echo +ABsAWw-93;42m+JYgliCWIJZMlkyWTJZIlkiWSJZElkSWR-...
-
About Virtual Terminal processing for version 6.3 and newer Most of the current command line utilities don't support Virtual Terminal processing yet. In this case ANSI escape sequences are not used to control the console output and their textual expressions get printed to the screen. Even if I expect that VT processing will be only barely used along with CONVERTCP, it won't hurt to enable it once that Windows 10 provides this possibility. Example using CONVERTCP v. 6.3: >nul chcp 65001 echo +ABsAWw-93;42m+JYgliCWIJZMlkyWTJZIlkiWSJZElkSWR-...
-
-
-
-
-
-
-
-
How to verify if the conversion to another encoding was correct? Beginning with version 6.0, CONVERTCP supports option /v which influence the return value of the utility. If you pass /v, CONVERTCP verifies whether all characters from the input have been converted without having used any replacement characters or approximated ASCII characters. Only in this case CONVERTCP returns 0. If one or more characters have been found that do not match the same Unicode code point in the used code page for the...
-
-
-
-
-
-
The new option /v (for "verify") makes CONVERTCP return 1 instead of 0 if at least one character has been replaced with a character that doesn't match the same Unicode code point. Besides of that it doesn't change the default behavior. That is, CONVERTCP silently replaces all characters that are invalid or that don't exist in the passed code pages with either a replacement character or with an approximated ASCII character which looks similar but has different semantics. Hopefully that will meet your...
-
-
-
-
-
Hannes, thank you very much for your feedback and suggestions! In general I decided to avoid the detection of the encoding. I explained the reasons in this Wiki post: https://sourceforge.net/p/convertcp/wiki/Home/#c4ac Actually your proposal is no encoding detection though. Even if you use it as such. Remember that ASCII characters and the related UTF-8 characters are absolutely equal and thus, you wouldn't even find any illegal characters. Also finding illegal characters might not necessarily mean...
-
Anonymous posted a comment on discussion Discussion about CONVERTCP
Hi Steffan - the author of CONVERTCP ! your convertcp ... 1.) ... is a beautifullest peace of software engineering! Thx. And thx for your sharing! 2.) My proposal - as I did a patch for testing: check if a file with CP_IN=UTF-8 is really a correct encoded file, see attachement. usage: CPCONVERT-MY.EXE UTF-8 0 /i "maybeUTF8file" /o NUL this will return an errorlevel=2, case the the "maybeUTF8file" is not an UTF-8 encoded file. (Beside the errorlevel, my slightly patch changes absolutely nothing of...
-
Thank you very much for your feedback Saso! Steffen
-
Anonymous posted a comment on discussion Discussion about CONVERTCP
Great tool! It is very useful for cases where old code pages are used (for example: conversion between CP852 and CP1250). Saso
-
-
-
-
You're welcome. Thanks again for your feedback. Much appreciated. Steffen
-
Anonymous posted a comment on discussion Discussion about CONVERTCP
Steffen: I apologize it took a week to get back to you to say thank you for explaining the process to remove the BOM. It makes perfect sense. Thanks again. Mike
-
Mike Glad to hear the utility is useful for you. Thank you for your feedback! Of course it is possible to remove the BOM from UTF-encoded text files. Just use the same ID or alias for both CP_In and CP_Out. If you omit option /b then the output file will not have the leading BOM anymore. Say, your file "test.txt" is UTF-8-encoded and has a BOM. In this case the command line would be like that: convertcp 65001 65001 /i "test.txt" /o "test_2.txt" "test_2.txt" will not contain the BOM. If you have any...
-
Michael Lee posted a comment on discussion Discussion about CONVERTCP
Steffen: I recently found CONVERTCP and am very happy I did, as it does a few things for me very well. I use it to convert characters with diacritics to the approximated ASCII characters, and to add BOM to certain files when needed. I don't see the ability to remove BOM. Is that possible? Many thanks, Mike
-
-
-
-
How to find the right Code Page ID and why CONVERTCP doesn't detect code pages automatically The supported code pages depend on the installed code pages on your computer. Option /l lists the installed code pages along with a short description. A lot of these IDs are not self-explanatory though. If you're familiar with .NET or HTML you may already know some MIME names that are listed on the Microsoft page https://msdn.microsoft.com/en-us/library/dd317756.aspx This table might be already helpful if...
-
-
-
-
-
-
-
-
How to find the right Code Page ID and why CONVERTCP doesn't detect code pages automatically The supported code pages depend on the installed code pages on your computer. Option /l lists the installed code pages along with a short description. A lot of these IDs are not self-explanatory though. If you're familiar with .NET or HTML you may already know some MIME names that are listed on the Microsoft page https://msdn.microsoft.com/en-us/library/dd317756.aspx This table might be already helpful if...
-
-
-
-
-
-
-
-
-
Threading in CONVERTCP explained As already written CONVERTCP reads the incoming stream chunk-wise if possible. The advantage is that an already converted chunk can be written using an asynchronuous thread at the same time as the next chunk of text is read and converted. That leads to a good performance. Furthermore the memory usage is limited to the buffer size the chunks need. This size doesn't increase even not if very large files are converted. Sounds like a good concept, doesn't it? That's the...
-
-
-
-
Option /n explained The /n is for "no threading". But what does it mean? For a better understanding let's step back to the defaults. As already written CONVERTCP reads the incoming stream chunk-wise. The advantage is that an already converted chunk can be written using an asynchronuous thread at the same time as the next chunk of text is read and converted. That leads to a good performance. Furthermore the memory usage is limited to the buffer size the chunks need. This size doesn't increase even...
-
-
-
-
All developers like to get feedback. So do I. For me, positive feedback is as valuable as negative feedback. I'd like to know about your experiences with CONVERTCP, found bugs, code reviews, suggestions etc. Because you have to sign in to create a ticket or to rate the project this forum is an additional platform where you are able to give feedback, also as an anonymous guest without having a SourceForge account. As long as spam bots don't exploit this opportunity I'll keep the forum open for everyone....
-
About flushing the output stream buffer Maybe you are wondering what the /f option is for. I have to admit you won't find much about it in the "readme.txt". Flushing is basically not needed. Written data will be buffered by the file system because the physical writing to the drive is slow. That way the performance can be improved enormously. There is no drawback as long as the file is not accessed concurrently. Appending new data to the buffer by another write operation will not corrupt the data...
-
About flushing the output stream Maybe you are wondering what the /f option is for. I have to admit you won't find much about it in the "readme.txt". Flushing is basically not needed. Written data will be buffered by the file system because the physical writing to the drive is slow. That way the performance can be improved enormously. There is no drawback as long as the file is not accessed concurrently. Appending new data to the buffer by another write operation will not corrupt the data in the...
-
CONVERTCP and non-ASCII file names The source code of a Batch script has to be ASCII, or at least encoded in the default OEM code page or the code page you have set using CHCP. However you'll get trouble to write file names that contain characters that are not supported. This is a limitation of the command interpreter. It's not a limitation of CONVERTCP as you may have observed while running my latest example above. So working with a wildcard character (like the asterisk) in a loop is already a good...
-
CONVERTCP and non-ASCII file names The source code of a Batch script has to be ASCII, or at least encoded encoded in the default OEM code page or the code page you have set using CHCP. However you'll get trouble to write file names that contain characters that are not supported. This is a limitation of the command interpreter. It's not a limitation of CONVERTCP as you may have observed while running my latest example above. So working with a wildcard character (like the asterisk) in a loop is already...
-
CONVERTCP and non-ASCII file names The source code of a Batch script have to be ASCII, or at least encoded encoded in the default OEM code page or the code page you have set using CHCP. However you'll get trouble to write file names that contain characters that are not supported. This is a limitation of the command interpreter. It's not a limitation of CONVERTCP as you may have observed while running my latest example above. So working with a wildcard character (like the asterisk) in a loop is already...
-
Convert all files of a certain directory Not the built-in options make CONVERTCP flexible to use. It's rather the command-line interface. You don't need to drag every single file to a window, you also don't need to browse files or folders in a dialog window. Just use the possibilities that the Windows command line already provide. Say, you want to convert all .txt files in the current directory from your default OEM code page to UTF-8 (similar to the example above) then you can just use a FOR loop...
-
Convert all files of a certain directory Not the built-in options make CONVERTCP flexible to use. It's rather the command-line interface. You don't need to drag every single file to a window, you also don't need to browse files or folders in a dialog window. Just use the possibilities that the windows command line already provide. Say, you want to convert all .txt files in the current directory from your default OEM code page to UTF-8 (similar to the example above) then you can just use a FOR loop...
-
Convert all files of a certain directory Not the build-in options make CONVERTCP flexible to use. It's rather the command-line interface. You don't need to drag every single file to a window, you also don't need to browse files or folders in a dialog window. Just use the possibilities that the windows command line already provide. Say, you want to convert all .txt files in the current directory from your default OEM code page to UTF-8 (similar to the example above) then you can just use a FOR loop...
-
Convert all files of a certain directory Not the build-in options make CONVERTCP flexible to use. It's rather the command-line interface. You don't need to drag every single file to a window, you also don't need to browse files or folders in a dialog window. Just use the possibilities that the windows command line already provide. Say, you want to convert all .txt files in the current directory from your default ANSI code page to UTF-8 (similar to the example above) then you can just use a FOR loop...
-
Replacing file content with the converted text I already wrote some examples in the "readme.txt" about how to convert a single file. You may ask why CONVERTCP doesn't support overwriting of a file with the converted content. The reason is that the tool doesn't read the entire content of the file at once. Especially for large files it's more memory-efficient to read only chunk-wise in order to avoid running out of RAM space. This also makes that CONVERTCP shows a pretty high performance because as...
-
-
-
Home
-
-
-
-
-