Leap2A import problem: "simplexml_load_file()... parser error : PCDATA invalid Char value..."

Using a for-loop and the PHP chr() command, I individually tested each ASCII character in the middle of an otherwise acceptable XML file. I tested them plain, after passing through htmlspecialchars(), and after passing through htmlentities(). Here is the list of the decimal integer codes for the ASCII characters that cause SimpleXML to choke. None of them are escaped by htmlspecialchars() or htmlentities().

$baddies = array(0,1,2,3,4,5,6,7,8,11,12,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31);

To help with testing (because it's not always easy to generate these characters), I've attached a file controlcharacters.txt which contains all 29 of these characters, in between <bad> tags. Depending on your text editor, opening it you may just see "<bad></bad>". But if you select the whole thing and paste it into a Mahara page title, you should be able to replicate the problem

To replicate:

1. Create a Mahara page with one or more of the forbidden characters in its page title
2. Export the page to Leap2a
3. Import the Leap2a file back into Mahara

Expected result: You've imported a copy of the page
Actual result: You get an error stack with the SimpleXML parser error as part of it.

On further research I've decided to be more thorough and just whitelist the allowed characters in XML, listed here: https://en.wikipedia.org/wiki/Valid_characters_in_XML

I wound up using preg_replace() with the "/u" modifier to make it Unicode-safe. The downside to this is that we read the entire file into memory and then do preg_replace on it, but that shouldn't use too much more memory, because we're already reading the entire file into memory in order to use simplexml.

I also discovered that htmlentities() can get rid of these invalid characters if you use the flags ENT_XML1 | ENT_DISALLOWED flags. But those flags were only added in PHP 5.4, and we still aim to support PHP 5.3. Plus, the best they can do is replace the invalid characters with a Unicode 0xFFDD character, which will display as an unprintable character. So, it's still better to just remove them entirely.

The patches for this had the wrong bug number on them.

Patch for master (15.10dev): https://reviews.mahara.org/#/c/5061/
Patch for 15.04: https://reviews.mahara.org/#/c/5080/