when is the html parser going to stop being a nazi

ok, news:

http://test.com/+does+this+woirk
http://test2.com/?+does+this+woirk

http://test.com/?googleresultsareuseful=does+this+woirk

ok there are your test cases

you know how an ILX URL is automatically converted to the thread title? Well, apparently " is converted to " :

Vi4gra "could help with Jetlag"...

oh you beat me

and it says we're illegal

NO SOMOS ILLEGAL

http://www.myspaceantics.com/images/funny/no-illegals-no-burritos.jpg

ah jw beat me to it

WHY CAN I ONLY POST 3 IMAGES U NAZIS

http://en.wikipedia.org/wiki/Capitol_Hill,_Seattle,_Washington,

http://images.amazon.com/images/P/0596002890.01.LZZZZZZZ.jpg

That last one is impossible to work out without additional markup, as it's not possible to work out whether it's a URL followed by a comma, or a URL with a comma in it. The starting point was the URL out of that book, but it's a long way off the mark.

Why not just break urls on white space?

I mean that'll still lead to badness if someone puts a , or ) after a url, but the onus would be on the user not the parser.

You could ignore punctuation if it's the last character of the url. That'd probably be correct for 99% of cases.

note to self: think more, post less

(?:http://(?:(?:(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:
[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z
])|(?:[A-Z]))|\d))\.)*(?:(?:(?:(?:[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|
(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z
]))))|(?:(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:(?
:(?:\d)+)))?)(?:/(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.
+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;:@&=])*)(?:/(?
:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:
%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;:@&=])*))*)(?:\?(?:(?:(?:(?:(?:(?:[a
-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:
[\dA-Fa-f])))|[;:@&=])*))?)?)|(?:ftp://(?:(?:(?:(?:(?:(?:(?:(?:[a-z])|
(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-
Fa-f])))|[;?&=])*)(?::(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$
_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;?&=])*))?@)?
(?:(?:(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:[a-z])|(?:
[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z])|(?:[A-Z
]))|\d))\.)*(?:(?:(?:(?:[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|(?:[A-Z]))
|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z]))))|(?:(
?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:(?:(?:\d)+))
)?))(?:/(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[
!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[?:@&=])*)(?:/(?:(?:(?:(?
:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-
Fa-f])(?:[\dA-Fa-f])))|[?:@&=])*))*)(?:;type=(?:[AIDaid]))?)?)|(?:news
:(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]
))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;/?:&=])+@(?:(?:(?:(?:(?:(?:(?:
(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:
[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z])|(?:[A-Z]))|\d))\.)*(?:(?:(?:(?:
[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|
(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z]))))|(?:(?:(?:\d)+)\.(?:(?:\d)+)\
.(?:(?:\d)+)\.(?:(?:\d)+))))|(?:(?:(?:[a-z])|(?:[A-Z]))(?:(?:(?:[a-z])
|(?:[A-Z]))|(?:\d)|[.+_-])*)|\*))|(?:nntp://(?:(?:(?:(?:(?:(?:(?:(?:(?
:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a
-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z])|(?:[A-Z]))|\d))\.)*(?:(?:(?:(?:[a
-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?
:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z]))))|(?:(?:(?:\d)+)\.(?:(?:\d)+)\.(
?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:(?:(?:\d)+)))?)/(?:(?:(?:[a-z])|(?:[A-
Z]))(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|[.+_-])*)(?:/(?:(?:\d)+))?)|(?:t
elnet://(?:(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(
?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;?&=])*)(?::(?:(?:(?:
(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\d
A-Fa-f])(?:[\dA-Fa-f])))|[;?&=])*))?@)?(?:(?:(?:(?:(?:(?:(?:(?:(?:[a-z
])|(?:[A-Z]))|\d)(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|
(?:[A-Z]))|\d))|(?:(?:(?:[a-z])|(?:[A-Z]))|\d))\.)*(?:(?:(?:(?:[a-z])|
(?:[A-Z]))(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z
]))|\d))|(?:(?:[a-z])|(?:[A-Z]))))|(?:(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:
\d)+)\.(?:(?:\d)+)))(?::(?:(?:(?:\d)+)))?))(?:/)?)|(?:gopher://(?:(?:(
?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:[a-z])|(?:[A-Z]))
|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z])|(?:[A-Z]))|\d)
)\.)*(?:(?:(?:(?:[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)
*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z]))))|(?:(?:(?:\d
)+)\.(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:(?:(?:\d)+)))?)(?:/
(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%
(?:[\dA-Fa-f])(?:[\dA-Fa-f]))|(?:[;/?:@&=])))(?:(?:(?:(?:(?:(?:[a-z])|
(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-
Fa-f]))|(?:[;/?:@&=]))*)(?:%09(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\
d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;:@&=
])*)(?:%09(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(
),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f]))|(?:[;/?:@&=]))*))?)?)?)?)|(?:(
?:wais://(?:(?:(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:[
a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z]
)|(?:[A-Z]))|\d))\.)*(?:(?:(?:(?:[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|(
?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z]
))))|(?:(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:(?:
(?:\d)+)))?)/(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!
*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))*))|(?:wais://(?:(?:(?:(?:(
?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-
)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z])|(?:[A-Z]))|\d))\.)*(
?:(?:(?:(?:[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?
:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z]))))|(?:(?:(?:\d)+)\.(
?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:(?:(?:\d)+)))?)/(?:(?:(?:
(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-F
a-f])(?:[\dA-Fa-f])))*)\?(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?
:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;:@&=])*))
|(?:wais://(?:(?:(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?
:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-
z])|(?:[A-Z]))|\d))\.)*(?:(?:(?:(?:[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])
|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-
Z]))))|(?:(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:(
?:(?:\d)+)))?)/(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:
[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))*)/(?:(?:(?:(?:(?:[a-z])|
(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-
Fa-f])))*)/(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'
(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))*)))|(?:mailto:(?:(?:(?:(?:(?
:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])
(?:[\dA-Fa-f]))|(?:[;/?:@&=]))+))|(?:file://(?:(?:(?:(?:(?:(?:(?:(?:(?
:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a
-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z])|(?:[A-Z]))|\d))\.)*(?:(?:(?:(?:[a
-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?
:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z]))))|(?:(?:(?:\d)+)\.(?:(?:\d)+)\.(
?:(?:\d)+)\.(?:(?:\d)+)))|localhost)?/(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[
A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f
])))|[?:@&=])*)(?:/(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+
-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[?:@&=])*))*))|(?
:prospero://(?:(?:(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(
?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a
-z])|(?:[A-Z]))|\d))\.)*(?:(?:(?:(?:[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z]
)|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A
-Z]))))|(?:(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:
(?:(?:\d)+)))?)/(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+
-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[?:@&=])*)(?:/(?:
(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%
(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[?:@&=])*))*)(?:(?:;(?:(?:(?:(?:(?:(?:[
a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?
:[\dA-Fa-f])))|[?:@&])*)=(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?
:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[?:@&])*)))
*)|(?:(?:(?:(?:[a-z])|(?:\d)|[+.-])+):(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))
|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f]))|(?
:[;/?:@&=]))*|(?://(?:(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?
:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;?&=])*)(?
::(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))
|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;?&=])*))?@)?(?:(?:(?:(?:(?:(?:(?
:(?:(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?
:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z])|(?:[A-Z]))|\d))\.)*(?:(?:(?
:(?:[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-
z])|(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z]))))|(?:(?:(?:\d)+)\.(?:(?:\d
)+)\.(?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:(?:(?:\d)+)))?))(?:/(?:(?:(?:(?:
(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f
])(?:[\dA-Fa-f]))|(?:[;/?:@&=]))*))?)))

(That mess is what it takes to fully regex a URL, btw. It's slow! And in Perl, too.)

Well you could probably ignore anything that isn't http...

(http://S+?)[-,:)]}=]? - the 99%, fuck user typos solution? ($1 being the link, final punc chars ignored). Ah it's still Perl specific though.

that worked well... should be //\S not //S.

(http://S+?)[-,:)}\]]?

And apparently Java has non-greedy matching so it should work.

except it doesn't.

Alright final attempt. Works in Java for http://en.wikipedia.org/wiki/Capitol_Hill,_Seattle,_Washington, when followed by whitespace or at the end of a line.

(http://\S+?)[-,:)}\]]\s*$

(//\S in the middle still no matter what it says above)

You could just use the [url] tags.

You could ignore punctuation if it's the last character of the url. That'd probably be correct for 99% of cases.

This is what I do.

I think the problem with a full auto-matching thing, is that, as we've discussed here before, it's just not possible. That's why markup languages exist. It's impossible to tell whether a space, or a comma, or a ], for example (http://www.google.com) etc. are part of a URL, or they're where it stops. This will even be true of the abomination above, which I assume was generated from EBNF. You have to make a call, and I've made one, which is what you're suggesting, as most URLs don't end with these (see the bracketed example above, which works, although it's actually unclear as to whether or not the final bracket should be included in the URL).

If your URL does look like that, we either annoy everyone my making it match, in which case the above bracketed example will fail and include the brackets in the URL, or you use the markup provided below to do it.

Anyway, the above examples have been sorted and work here:

http://www.ilxor.com:8090/ILX/ThreadSelectedControllerServlet?boardid=159&threadid=21

(?:http://(?:(?:(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:
[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z
])|(?:[A-Z]))|\d))\.)*(?:(?:(?:(?:[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|
(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z
]))))|(?:(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:(?
:(?:\d)+)))?)(?:/(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.
+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;:@&=])*)(?:/(?
:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:
%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;:@&=])*))*)(?:\?(?:(?:(?:(?:(?:(?:[a
-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:
[\dA-Fa-f])))|[;:@&=])*))?)?)|(?:ftp://(?:(?:(?:(?:(?:(?:(?:(?:[a-z])|
(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-
Fa-f])))|[;?&=])*)(?::(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$
_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;?&=])*))?@)?
(?:(?:(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:[a-z])|(?:
[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z])|(?:[A-Z
]))|\d))\.)*(?:(?:(?:(?:[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|(?:[A-Z]))
|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z]))))|(?:(
?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:(?:(?:\d)+))
)?))(?:/(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[
!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[?:@&=])*)(?:/(?:(?:(?:(?
:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-
Fa-f])(?:[\dA-Fa-f])))|[?:@&=])*))*)(?:;type=(?:[AIDaid]))?)?)|(?:news
:(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]
))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;/?:&=])+@(?:(?:(?:(?:(?:(?:(?:
(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:
[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z])|(?:[A-Z]))|\d))\.)*(?:(?:(?:(?:
[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|
(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z]))))|(?:(?:(?:\d)+)\.(?:(?:\d)+)\
.(?:(?:\d)+)\.(?:(?:\d)+))))|(?:(?:(?:[a-z])|(?:[A-Z]))(?:(?:(?:[a-z])
|(?:[A-Z]))|(?:\d)|[.+_-])*)|\*))|(?:nntp://(?:(?:(?:(?:(?:(?:(?:(?:(?
:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a
-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z])|(?:[A-Z]))|\d))\.)*(?:(?:(?:(?:[a
-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?
:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z]))))|(?:(?:(?:\d)+)\.(?:(?:\d)+)\.(
?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:(?:(?:\d)+)))?)/(?:(?:(?:[a-z])|(?:[A-
Z]))(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|[.+_-])*)(?:/(?:(?:\d)+))?)|(?:t
elnet://(?:(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(
?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;?&=])*)(?::(?:(?:(?:
(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\d
A-Fa-f])(?:[\dA-Fa-f])))|[;?&=])*))?@)?(?:(?:(?:(?:(?:(?:(?:(?:(?:[a-z
])|(?:[A-Z]))|\d)(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|
(?:[A-Z]))|\d))|(?:(?:(?:[a-z])|(?:[A-Z]))|\d))\.)*(?:(?:(?:(?:[a-z])|
(?:[A-Z]))(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z
]))|\d))|(?:(?:[a-z])|(?:[A-Z]))))|(?:(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:
\d)+)\.(?:(?:\d)+)))(?::(?:(?:(?:\d)+)))?))(?:/)?)|(?:gopher://(?:(?:(
?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:[a-z])|(?:[A-Z]))
|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z])|(?:[A-Z]))|\d)
)\.)*(?:(?:(?:(?:[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)
*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z]))))|(?:(?:(?:\d
)+)\.(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:(?:(?:\d)+)))?)(?:/
(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%
(?:[\dA-Fa-f])(?:[\dA-Fa-f]))|(?:[;/?:@&=])))(?:(?:(?:(?:(?:(?:[a-z])|
(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-
Fa-f]))|(?:[;/?:@&=]))*)(?:%09(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\
d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;:@&=
])*)(?:%09(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(
),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f]))|(?:[;/?:@&=]))*))?)?)?)?)|(?:(
?:wais://(?:(?:(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:[
a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z]
)|(?:[A-Z]))|\d))\.)*(?:(?:(?:(?:[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|(
?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z]
))))|(?:(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:(?:
(?:\d)+)))?)/(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!
*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))*))|(?:wais://(?:(?:(?:(?:(
?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-
)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z])|(?:[A-Z]))|\d))\.)*(
?:(?:(?:(?:[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?
:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z]))))|(?:(?:(?:\d)+)\.(
?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:(?:(?:\d)+)))?)/(?:(?:(?:
(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-F
a-f])(?:[\dA-Fa-f])))*)\?(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?
:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;:@&=])*))
|(?:wais://(?:(?:(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?
:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-
z])|(?:[A-Z]))|\d))\.)*(?:(?:(?:(?:[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])
|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-
Z]))))|(?:(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:(
?:(?:\d)+)))?)/(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:
[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))*)/(?:(?:(?:(?:(?:[a-z])|
(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-
Fa-f])))*)/(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'
(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))*)))|(?:mailto:(?:(?:(?:(?:(?
:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])
(?:[\dA-Fa-f]))|(?:[;/?:@&=]))+))|(?:file://(?:(?:(?:(?:(?:(?:(?:(?:(?
:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a
-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z])|(?:[A-Z]))|\d))\.)*(?:(?:(?:(?:[a
-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?
:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z]))))|(?:(?:(?:\d)+)\.(?:(?:\d)+)\.(
?:(?:\d)+)\.(?:(?:\d)+)))|localhost)?/(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[
A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f
])))|[?:@&=])*)(?:/(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+
-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[?:@&=])*))*))|(?
:prospero://(?:(?:(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(
?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a
-z])|(?:[A-Z]))|\d))\.)*(?:(?:(?:(?:[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z]
)|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A
-Z]))))|(?:(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:
(?:(?:\d)+)))?)/(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+
-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[?:@&=])*)(?:/(?:
(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%
(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[?:@&=])*))*)(?:(?:;(?:(?:(?:(?:(?:(?:[
a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?
:[\dA-Fa-f])))|[?:@&])*)=(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?
:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[?:@&])*)))
*)|(?:(?:(?:(?:[a-z])|(?:\d)|[+.-])+):(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))
|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f]))|(?
:[;/?:@&=]))*|(?://(?:(?:(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?
:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;?&=])*)(?
::(?:(?:(?:(?:(?:(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))
|(?:%(?:[\dA-Fa-f])(?:[\dA-Fa-f])))|[;?&=])*))?@)?(?:(?:(?:(?:(?:(?:(?
:(?:(?:[a-z])|(?:[A-Z]))|\d)(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?
:(?:[a-z])|(?:[A-Z]))|\d))|(?:(?:(?:[a-z])|(?:[A-Z]))|\d))\.)*(?:(?:(?
:(?:[a-z])|(?:[A-Z]))(?:(?:(?:(?:[a-z])|(?:[A-Z]))|\d)|-)*(?:(?:(?:[a-
z])|(?:[A-Z]))|\d))|(?:(?:[a-z])|(?:[A-Z]))))|(?:(?:(?:\d)+)\.(?:(?:\d
)+)\.(?:(?:\d)+)\.(?:(?:\d)+)))(?::(?:(?:(?:\d)+)))?))(?:/(?:(?:(?:(?:
(?:[a-z])|(?:[A-Z]))|(?:\d)|(?:[$_.+-])|(?:[!*'(),]))|(?:%(?:[\dA-Fa-f
])(?:[\dA-Fa-f]))|(?:[;/?:@&=]))*))?)))

-- stet, Wednesday, May 30, 2007 2:10 AM (4 hours ago) Bookmark Link

--------------------------------------------------------------------------------

(That mess is what it takes to fully regex a URL, btw. It's slow! And in Perl, too.)

-- stet, Wednesday, May 30, 2007 2:14 AM (4 hours ago) Bookmark Link

Purely a coincidence, but that is also the secret "Myspace Password Key" which can be entered in the password field for any account whoose owner has lost thier myspace password.

Guys why are you writing your own url parser

Hi Jon,

I guess the starting point, as always, was to look for something that already did it, but the problem you find, especially if restricting yourself to just open source stuff, is how do you know that what you find is the ideal solution? I looked at quite a lot of them (and started by using one), but they were all rubbish and took two minutes to think of cases that wouldn't work with them. Everyone in open source thinks their solution is best one that's ever existed - the open source world is stuffed full of dicks trying to further their own careers by getting a name for themselves, so it's hard to sort the wheat from the chaff.

I found the same problem with quite a lot of other stuff... Caches, for example, don't document if they're synchronized or not, or if they are, they're usually fully synchronized, rather than read/write locked, which leads to lower throughput etc. So I wound up writing a bunch of that myself. The first URL parser I had, was something I took from something else, but it was rubbish, as you and others pointed out, so I totally revamped it, and that's been much better. I log all the successes/failures to see how well it does, as well as look at the site so see how it does and lately, the only problems I've seen are the stuff you've posted, which I've sorted (took two minutes), so unless loads more problems turn up, I'm not sure there is a problem to be addressed. If you've got something that you could describe as the best regular expression ever for parsing URLs, then post it and I'll use it.

It is an interesting question, in that, at work, I would pretty much always recommend using something off-the-shelf, but in doing this, I've come to realise that (arrogant as it may sound) that I have more experience most of the open source guys writing the off-the-shelf stuff, or maybe more crucially, no way of telling for sure whether I'm better at it than they are, so writing ILX has kind of changed my opinion on this sort of thing. This doesn't apply at work, because there are loads of people involved, and often they don't have the experiences of the open source guys, so using even substandard open source stuff is an overall win.

If you weren't using BBCode there's all kinds of xml transforms that can make this way easier

let go

I don't follow. I can't think of any way in which an XML transform (I assume you mean XSLT transforming an XML document) can possibly help you in automatically identifying hyperlinks in a document when they're not marked up.

Yea, that point isn't really related. :/

ok so urls with @ signs in them get fucked. even if the @ is in the path component

http://www.youtube.com/watch?v=BzIT6rzkzok

^ doesn't convert using bbcode button right

I see what you mean about the @ signs. The URL code does the right thing, but putting an @ in it makes it look like an email address, so the email obscuring code goes and plasters *s all over it.

http://✧✧✧.t✧✧✧@t✧✧✧.c✧✧

also probably a good idea to filter out @ signs for username/pw in url

Do you not want to be able to do something like an anonymous sftp link? Or is that not what you mean?

Another subject - do you know how to style radio buttons and checkboxes with CSS? I think I'm doing the right thing, but it's not going for it. Buttons were easy enough, but not the others. I've got a black stylesheet (like a terminal emulator), but they wind up with these big white blobs all over them when it's a poll and so on...

Although Safari doesn't like having anything done to its buttons either.

Safari allows you to style the buttons, but it drops them out of Aqua and makes them just plain boxes if you try it.

Hmm - not working for me. Works in Firefox.

Have a look on the sandbox, if you like. Stylesheet's there.

Looks like you just can't do it...

ooh, radio buttons, sorry. You can't style them, no.

http://www.thedailywtf.com/Images/200706/error'd/204.png

I can't post this image because YOU HAVEN'T FIXED THIS

HELLO

http://forumserver.twoplustwo.com/showflat.php?Cat=0&Number=11034555&;page=0&fpart=1&vc=1

^ hello

PEBKAC

rong

The point was that you're not supposed to stop url parsing on a semicolon

moron

nerrr

The explanation for this is just the same explanation as I gave upthread when you asked this question before, but for a comma and not a semicolon:

when is the html parser going to stop being a nazi

hey how about a wee comment (like it used to be, when you messed up your html tags) if a post contains common html tags, saying 'did you mean to use bb code?'

I've been meaning to add that to my bbcode checker stuff. At least a warning on '<' followed by an alphabetic character. Work kinda sucks now, tho.

http://en.wikipedia.org/wiki/Iron_maiden_(torture_device)

http://lucasarts.vintagegaming.org/limages/screenshots/loom/fmtowns/44_-_The_swan_turns_out_to_be_Bobbin's_mother_(FM_Towns).gif

doesn't work in img tag

http://answers.yahoo.com/question/index;_ylt=AhmeNPo56wkvdyeJmmgnNXHK7BR.;_ylv=3?qid=20071128113104AASaBbN

^ doesn't work in url tag

SERIOSULKY

already posted :*(

HEY THE URL PARSER BLOWS

www.blacklight.com/backroom/backroom.nsf/allitems/52A92B7091B0ECF18625719B006BB2FA/$file/FSCENT.gif

http://www.blacklight.com/backroom/backroom.nsf/allitems/52A92B7091B0ECF18625719B006BB2FA/

who the fuck puts $file in a URL

people i still bandwidth from

so we have HIDDEN BROADS but the html parser is still a nazi

way to go

go us!!

http://media.monstersandcritics.com/articles/1256801/article_images/nerds.jpg

Dude, there is no HTML parser.

who doesn't have a hidden broad?

for the record, i'm happy there's a vidya game bored. Bewteen that and the main site, i have few other board-needs.

So start whatever boards ya like; go knock yerself out.