Characters in Blogs Odd

Dazed
@dazed
12 years ago
1,022 posts

I am starting to see these pop up all over after the release to 5.1. The below is a cut and paste from 1. I had one in mine also where I just had a "Ã". Mine was not a cut/paste but a few lines of typed info.

Ã�ï¿½Ã¯Â¿Â½Ã�ï¿½Ã�Â
updated by @dazed: 12/14/13 01:02:17PM

@brian
12 years ago
10,148 posts

How can I reproduce this?

Thanks!

--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

Dazed
@dazed
12 years ago
1,022 posts

I wish I could tell you! I am researching.

@brian
12 years ago
10,148 posts

Yeah I've not seen this in testing here - I'm going to guess users are cutting and pasting into the editor, and what we're seeing is UTF8 codes for the weird characters.

--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

Dazed
@dazed
12 years ago
1,022 posts

That blog was typed in directly.

@brian
12 years ago
10,148 posts

So just to be clear, you're typing in normal characters into the blog, but after saving you are getting a bunch of garbage?

--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

Dazed
@dazed
12 years ago
1,022 posts

Correct. I emailed the member and he said he just typed it in. So he did not see it in the editor, but it appears that way on the blog. I have seen it once or twice in my blogs but never to that extent.

@michael
12 years ago
7,772 posts

take a look at the text directly in the database.

Whether its readable there (phpmyadmin) or not could help indicate the location of the problem.

Dazed
@dazed
12 years ago
1,022 posts

Thanks Michael I will take a look.

@brian
12 years ago
10,148 posts

I looked at them in the DB and they are all character entities - I just have no idea how those are able to be placed into the blog entry without the user entering them. Let me know if you find a way to reproduce it.

--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

@michael
12 years ago
7,772 posts

If they are that way in the DB it points to copy+paste of the content and user input.

changing this thread to japanese encoding shift-jis from the browser options makes them change into japanese characters, but its nothing intelligible.

Dazed
@dazed
12 years ago
1,022 posts

Hey guys I was woeking with Brian on this and he could not duplicate it. I can reproduce it on Win 7 Chrome and Safari and Mac Chrome - http://test.jamroom.net/dazed/blog

Can you guys try these steps:

1. Create a new blog with some basic text (no cut/paste)
2. submit
3. edit the blog - click tab and go to the next line (you may or may not see the char yet)
4. add some more text on the next line
5. submit
6. edit again

Result - Usually by the third time I see the weird char appear.
updated by @dazed: 11/03/13 04:23:47AM

@brian
12 years ago
10,148 posts

Yeah I tested this with Dazed, but was not able to replicate on my Mac (FF, Chrome and Safari) or my windows 7 machine (FF, IE and Chrome).

Let me know if anyone else can replicate.

Thanks!

--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

@douglas
12 years ago
2,797 posts

I just tried this and could not get the character to show.

Using FF24.0.

--

Douglas Hackney
Jamroom Team - Designer/Developer/Support
FAQ-Docs-Help Videos

Dazed
@dazed
12 years ago
1,022 posts

ok this is bothering the hell out out of me. I am using utf-8. If I go to any page I can usually find a string of odd chars. If I edit a blog that does not display the char, in edit mode it is there.

Suggestions?

@douglas
12 years ago
2,797 posts

That is odd, the only time I've ever seen characters like that is from pasting text from Notepad or Word.

--

Douglas Hackney
Jamroom Team - Designer/Developer/Support
FAQ-Docs-Help Videos

Dazed
@dazed
12 years ago
1,022 posts

Joomla had the issue as well related to PHP 5.4.

Summary
get_html_translation_table requires encoding value for PHP 5.4
Details
The get_html_translation_table function used by the JFilterInput::_decode method requires setting the encoding value to "ISO-8859-1" as this value is set to "UTF-8" by default in PHP 5.4 - http://php.net/manual/en/function.get-html-translation-table.php

http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_id=11410&tracker_item_id=29944

@brian
12 years ago
10,148 posts

Jamroom does not use get_html_translation_table, so I don't believe that is involved.

--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

@brian
12 years ago
10,148 posts

Has anyone else been able to replicate this issue?

Thanks!

--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

@brian
12 years ago
10,148 posts

Dazed - can you test it out again here and see if you still see the issue? I added in the entity encoding values to the htmlentities() function calls that are used on the form values - not sure if that is going to work or not, but it's worth a try. If that DOES work, it's strange that it was only affecting you, as it should have affected us all.

Let me know.

Thanks!

--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

SteveX
@ultrajam
12 years ago
2,584 posts

I tried earlier but everything was fine after several edits and saves.

--
¯\_(ツ)_/¯ Education, learning resources, TEL, AR/VR/MR, CC licensed content, panoramas, interactive narrative, sectional modules (like jrDocs), lunch at Uni of Bristol. Get in touch if you share my current interests or can suggest better :)

Dazed
@dazed
12 years ago
1,022 posts

Brian - I just tried on here with mac safari, win 7 ff and chrome and could not duplicate.

@brian
12 years ago
10,148 posts

Well that's good news, but a bit perplexing

I'll get the update for the Core module pushed out and you can test it on your system and let me know if it fixes it there for you.

I just don't understand why it only affected you, which is strange.

Thanks!

--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

Dazed
@dazed
12 years ago
1,022 posts

I agree 100%. From the research I did it all pointed to entity encoding and my site was set to utf8 in the meta.tpl and db should be utf8_general_ci from what I remember. So there should be no conflicts there. But it definitely was effecting a lot on my site.

Thanks for not giving up on this as user error

@brian
12 years ago
10,148 posts

Yeah no problem - everything in JR5 is UTF-8 all the way through, and I do see they made a change in PHP 5.4 to make it also be UTF-8 by default (5.3 is ISO-8859-1), but since that's all "server side" I'm not sure why we did not see it.

Anyway, I've uploaded an update jrCore-5.1.0 - of you reload that from your Marketplace -> Tools -> Reload modules and skins section, it should fix it on your site.

Hope this helps!

--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

Dazed
@dazed
12 years ago
1,022 posts

I just updated. I ran tests before and could reproduce the issue and could not after the fix. Thanks for taking care of this.

@brian
12 years ago
10,148 posts

Awesome - glad to hear it

--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

@douglas
12 years ago
2,797 posts

brian:
Dazed - can you test it out again here and see if you still see the issue? I added in the entity encoding values to the htmlentities() function calls that are used on the form values - not sure if that is going to work or not, but it's worth a try. If that DOES work, it's strange that it was only affecting you, as it should have affected us all.

Let me know.

Thanks!

Would this change cause this php notice?

[03-Nov-2013 13:21:46 America/Vancouver] PHP Notice: Use of undefined constant ENT_HTML401 - assumed 'ENT_HTML401' in /home/pekard/public_html/Jamroom5/modules/jrCore/lib/form.php on line 374 [x 4]

--

Douglas Hackney
Jamroom Team - Designer/Developer/Support
FAQ-Docs-Help Videos

@brian
12 years ago
10,148 posts

yeah that would - what version of PHP are you running?

Thanks!

--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

@brian
12 years ago
10,148 posts

Looks like ENT_HTML401 was added in PHP 5.4.0, so you're probably on 5.3 - let me get that fixed up and a new core pushed out.

--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

@brian
12 years ago
10,148 posts

Okay reload the core and this should be good to go now.

Let me know if you see any issues.

Thanks!

--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net

Dazed
@dazed
12 years ago
1,022 posts

I am on 5.3.10-1ubuntu3.8

@douglas
12 years ago
2,797 posts

I'm getting this php error when I try to reload the jrCore module.

[04-Nov-2013 05:17:18 America/Vancouver] PHP Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 22393857 bytes) in /home/pekard/public_html/Jamroom5/modules/jrCore/lib/util.php on line 872 [x 1]

--

Douglas Hackney
Jamroom Team - Designer/Developer/Support
FAQ-Docs-Help Videos

@blindmime
12 years ago
772 posts

My system is doing the same thing as Dazed. I believe we're both on jbservers. Is the core safe to reload (re: Douglas' error)?

Dazed
@dazed
12 years ago
1,022 posts

Hey Blindmime I am no longer with JBServers and thank you for confirming you had the issue also. I was feeling alone lol. Brian's fix resolved this issue and I am not seeing the error Douglas is seeing.
updated by @dazed: 11/04/13 06:45:01AM

@brian
12 years ago
10,148 posts

blindmime:
My system is doing the same thing as Dazed. I believe we're both on jbservers. Is the core safe to reload (re: Douglas' error)?

Yes - you should have no problem going into Marketplace -> Tools -> Reload Modules and Skins and click on "reload" for the System Core, and that will update you.

Hope this helps!

--
Brian Johnson
Founder and Lead Developer - Jamroom
https://www.jamroom.net