<browserCaps> and NULL UserAgent

  • Thread starter Thread starter Jason Collins
  • Start date Start date
J

Jason Collins

There are some Requests that occur that have a NULL UserAgent. Some of these
are legitimate crawlers (e.g., Overture's crawler), and I'm attempting to
adjust my <browserCaps> to correctly classify crawlers.

In an attempt to capture browsers with (UserAgent == null), I've added a
case like this:

<filter>
<case match="^$">
crawler=true
</case>
</filter>

But that only seems to trap the empty string (as expected, really).

Does anyone know a way to match the null string?

Thanks,
j
 
Hi Jason,

Thank you for posting in community.
From your description, you're wondering how to set the browserCap' filter
so as to capture those clients whose useragent is null, yes?
Regarding on the issue, I am
finding proper resource to assist you and we will update as soon as posible.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security(This posting is provided "AS IS",
with no warranties, and confers no rights.)
 
Hi Jason,

I am currently researching your issue and attempting to repro. I will post
back here ASAP. It will be sometime tomorrow.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.


--------------------
From: "Jason Collins" <[email protected]>
References: <[email protected]>
Click to expand...
 
Hi Jason,

I'm still researching this. It's going to be a bit longer before I'll have
something for you.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.

--------------------
X-Tomcat-ID: 749098030
References: <[email protected]>
Click to expand...
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
From: (e-mail address removed) (Jim Cheshire [MSFT])
Organization: Microsoft
Date: Mon, 23 Feb 2004 23:56:52 GMT
Subject: Re: <browserCaps> and NULL UserAgent
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet
Message-ID: <CRta5im#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
Lines: 47
Path: cpmsftngxa06.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:213135
NNTP-Posting-Host: tomcatimport2.phx.gbl 10.201.218.182

Hi Jason,

I am currently researching your issue and attempting to repro. I will post
back here ASAP. It will be sometime tomorrow.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.


--------------------
From: "Jason Collins" <[email protected]>
References: <[email protected]>
Click to expand...
Subject: Re: <browserCaps> and NULL UserAgent
Date: Mon, 23 Feb 2004 08:54:02 -0600
Lines: 24
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
Message-ID: <#DIRhzh#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
NNTP-Posting-Host: codemonkey.net 142.165.166.48
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP11.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:212973
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet

Yes, this is what I want to do (i.e., match clients with UserAgent==null)
j
Click to expand...
Click to expand...
 
Hi Jason,

I have conferred with all of the experts, and the bottom line is that this
cannot be done. You can match an empty string, but you cannot match
something that is undefined. In order to match something with RegEx, you
must first define it.

In the process of troubleshooting your issue, I was pointed at a really
cool tool that I thought I'd share with you. Here it is:

http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=C712F
2DF-B026-4D58-8961-4EE2729D7322

Thanks.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.

--------------------
X-Tomcat-ID: 713102383
References: <[email protected]>
Click to expand...
<[email protected]>
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
From: (e-mail address removed) (Jim Cheshire [MSFT])
Organization: Microsoft
Date: Tue, 24 Feb 2004 23:25:19 GMT
Subject: Re: <browserCaps> and NULL UserAgent
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet
Message-ID: <l7L351y#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
Lines: 75
NNTP-Posting-Host: TOMCATIMPORT1 10.201.218.122
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGXA06.phx.gbl!cpmsftngxa10.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:213409

Hi Jason,

I'm still researching this. It's going to be a bit longer before I'll have
something for you.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.

--------------------
X-Tomcat-ID: 749098030
References: <[email protected]>
Click to expand...
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
From: (e-mail address removed) (Jim Cheshire [MSFT])
Organization: Microsoft
Date: Mon, 23 Feb 2004 23:56:52 GMT
Subject: Re: <browserCaps> and NULL UserAgent
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet
Message-ID: <CRta5im#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
Lines: 47
Path: cpmsftngxa06.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:213135
NNTP-Posting-Host: tomcatimport2.phx.gbl 10.201.218.182

Hi Jason,

I am currently researching your issue and attempting to repro. I will post
back here ASAP. It will be sometime tomorrow.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.


--------------------
From: "Jason Collins" <[email protected]>
References: <[email protected]>
Click to expand...
Subject: Re: <browserCaps> and NULL UserAgent
Date: Mon, 23 Feb 2004 08:54:02 -0600
Lines: 24
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
Message-ID: <#DIRhzh#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
NNTP-Posting-Host: codemonkey.net 142.165.166.48
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP11.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:212973
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet

Yes, this is what I want to do (i.e., match clients with UserAgent==null)
j

Hi Jason,

Thank you for posting in community.
From your description, you're wondering how to set the browserCap' filter
so as to capture those clients whose useragent is null, yes?
Regarding on the issue, I am
finding proper resource to assist you and we will update as soon as
posible.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security(This posting is provided "AS IS",
with no warranties, and confers no rights.)
Click to expand...
Click to expand...
Click to expand...
 
Perhaps I'm missing something. Why not just test the
Request.Headers["User-Agent"] against null?

--
Thanks,

Eric Lawrence
Program Manager
Assistance and Worldwide Services

This posting is provided "AS IS" with no warranties, and confers no rights.

Jim Cheshire said:
Hi Jason,

I have conferred with all of the experts, and the bottom line is that this
cannot be done. You can match an empty string, but you cannot match
something that is undefined. In order to match something with RegEx, you
must first define it.

In the process of troubleshooting your issue, I was pointed at a really
cool tool that I thought I'd share with you. Here it is:

http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=C712F
2DF-B026-4D58-8961-4EE2729D7322

Thanks.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.

--------------------
X-Tomcat-ID: 713102383
References: <[email protected]>
Click to expand...
<[email protected]>
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
From: (e-mail address removed) (Jim Cheshire [MSFT])
Organization: Microsoft
Date: Tue, 24 Feb 2004 23:25:19 GMT
Subject: Re: <browserCaps> and NULL UserAgent
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet
Message-ID: <l7L351y#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
Lines: 75
NNTP-Posting-Host: TOMCATIMPORT1 10.201.218.122
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGXA06.phx.gbl!cpmsftngxa10.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:213409

Hi Jason,

I'm still researching this. It's going to be a bit longer before I'll have
something for you.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.

--------------------
X-Tomcat-ID: 749098030
References: <[email protected]>
Click to expand...
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
From: (e-mail address removed) (Jim Cheshire [MSFT])
Organization: Microsoft
Date: Mon, 23 Feb 2004 23:56:52 GMT
Subject: Re: <browserCaps> and NULL UserAgent
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet
Message-ID: <CRta5im#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
Lines: 47
Path: cpmsftngxa06.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:213135
NNTP-Posting-Host: tomcatimport2.phx.gbl 10.201.218.182

Hi Jason,

I am currently researching your issue and attempting to repro. I will post
back here ASAP. It will be sometime tomorrow.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.


--------------------
From: "Jason Collins" <[email protected]>
References: <[email protected]>
<[email protected]>
Subject: Re: <browserCaps> and NULL UserAgent
Date: Mon, 23 Feb 2004 08:54:02 -0600
Lines: 24
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
Message-ID: <#DIRhzh#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
NNTP-Posting-Host: codemonkey.net 142.165.166.48
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP11.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:212973
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet

Yes, this is what I want to do (i.e., match clients with UserAgent==null)
j

Hi Jason,

Thank you for posting in community.
From your description, you're wondering how to set the browserCap' filter
so as to capture those clients whose useragent is null, yes?
Regarding on the issue, I am
finding proper resource to assist you and we will update as soon as
posible.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security(This posting is provided "AS IS",
with no warranties, and confers no rights.)
Click to expand...
Click to expand...
Click to expand...
 
Because I'm want to do this centrally in the machine.config's <browserCaps>
so that it doesn't litter my code all over the place.

j


Eric Lawrence said:
Perhaps I'm missing something. Why not just test the
Request.Headers["User-Agent"] against null?

--
Thanks,

Eric Lawrence
Program Manager
Assistance and Worldwide Services

This posting is provided "AS IS" with no warranties, and confers no rights.

Jim Cheshire said:
Hi Jason,

I have conferred with all of the experts, and the bottom line is that this
cannot be done. You can match an empty string, but you cannot match
something that is undefined. In order to match something with RegEx, you
must first define it.

In the process of troubleshooting your issue, I was pointed at a really
cool tool that I thought I'd share with you. Here it is:
Click to expand...
http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=C712F
2DF-B026-4D58-8961-4EE2729D7322

Thanks.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.

--------------------
X-Tomcat-ID: 713102383
References: <[email protected]>
Click to expand...
<[email protected]>
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
From: (e-mail address removed) (Jim Cheshire [MSFT])
Organization: Microsoft
Date: Tue, 24 Feb 2004 23:25:19 GMT
Subject: Re: <browserCaps> and NULL UserAgent
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet
Message-ID: <l7L351y#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
Lines: 75
NNTP-Posting-Host: TOMCATIMPORT1 10.201.218.122
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGXA06.phx.gbl!cpmsftngxa10.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:213409

Hi Jason,

I'm still researching this. It's going to be a bit longer before I'll have
something for you.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.

--------------------
X-Tomcat-ID: 749098030
References: <[email protected]>
<[email protected]>
<#DIRhzh#[email protected]>
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
From: (e-mail address removed) (Jim Cheshire [MSFT])
Organization: Microsoft
Date: Mon, 23 Feb 2004 23:56:52 GMT
Subject: Re: <browserCaps> and NULL UserAgent
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet
Message-ID: <CRta5im#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
Lines: 47
Path: cpmsftngxa06.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:213135
NNTP-Posting-Host: tomcatimport2.phx.gbl 10.201.218.182

Hi Jason,

I am currently researching your issue and attempting to repro. I will
post
back here ASAP. It will be sometime tomorrow.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.


--------------------
From: "Jason Collins" <[email protected]>
References: <[email protected]>
<[email protected]>
Subject: Re: <browserCaps> and NULL UserAgent
Date: Mon, 23 Feb 2004 08:54:02 -0600
Lines: 24
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
Message-ID: <#DIRhzh#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
NNTP-Posting-Host: codemonkey.net 142.165.166.48
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP11.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:212973
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet

Yes, this is what I want to do (i.e., match clients with UserAgent==null)
j

Hi Jason,

Thank you for posting in community.
From your description, you're wondering how to set the browserCap'
filter
so as to capture those clients whose useragent is null, yes?
Regarding on the issue, I am
finding proper resource to assist you and we will update as soon as
posible.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security(This posting is provided "AS IS",
with no warranties, and confers no rights.)
Click to expand...
Click to expand...
Click to expand...
 
Jason,

That's just not going to be possible via the machine.config file. There is
no Regular Expression that will match a value that is not defined.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.

--------------------
From: "Jason Collins" <[email protected]>
References: <[email protected]>
Click to expand...
<[email protected]>
<#DIRhzh#[email protected]>
<CRta5im#[email protected]>
<l7L351y#[email protected]>
Subject: Re: <browserCaps> and NULL UserAgent
Date: Thu, 26 Feb 2004 16:23:54 -0600
Lines: 175
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
Message-ID: <[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
NNTP-Posting-Host: codemonkey.net 142.165.166.48
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP09.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:214088
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet

Because I'm want to do this centrally in the machine.config's <browserCaps>
so that it doesn't litter my code all over the place.

j


Eric Lawrence said:
Perhaps I'm missing something. Why not just test the
Request.Headers["User-Agent"] against null?

--
Thanks,

Eric Lawrence
Program Manager
Assistance and Worldwide Services

This posting is provided "AS IS" with no warranties, and confers no rights.

Jim Cheshire said:
Hi Jason,

I have conferred with all of the experts, and the bottom line is that this
cannot be done. You can match an empty string, but you cannot match
something that is undefined. In order to match something with RegEx, you
must first define it.

In the process of troubleshooting your issue, I was pointed at a really
cool tool that I thought I'd share with you. Here it is:
Click to expand...
http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=C712 F
2DF-B026-4D58-8961-4EE2729D7322

Thanks.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.

--------------------
X-Tomcat-ID: 713102383
References: <[email protected]>
<[email protected]>
<#DIRhzh#[email protected]>
<CRta5im#[email protected]>
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
From: (e-mail address removed) (Jim Cheshire [MSFT])
Organization: Microsoft
Date: Tue, 24 Feb 2004 23:25:19 GMT
Subject: Re: <browserCaps> and NULL UserAgent
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet
Message-ID: <l7L351y#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
Lines: 75
NNTP-Posting-Host: TOMCATIMPORT1 10.201.218.122
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGXA06.phx.gbl!cpmsftngxa10.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:213409

Hi Jason,

I'm still researching this. It's going to be a bit longer before I'll
have
something for you.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.

--------------------
X-Tomcat-ID: 749098030
References: <[email protected]>
<[email protected]>
<#DIRhzh#[email protected]>
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
From: (e-mail address removed) (Jim Cheshire [MSFT])
Organization: Microsoft
Date: Mon, 23 Feb 2004 23:56:52 GMT
Subject: Re: <browserCaps> and NULL UserAgent
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet
Message-ID: <CRta5im#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
Lines: 47
Path: cpmsftngxa06.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:213135
NNTP-Posting-Host: tomcatimport2.phx.gbl 10.201.218.182

Hi Jason,

I am currently researching your issue and attempting to repro. I will
post
back here ASAP. It will be sometime tomorrow.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.


--------------------
From: "Jason Collins" <[email protected]>
References: <[email protected]>
<[email protected]>
Subject: Re: <browserCaps> and NULL UserAgent
Date: Mon, 23 Feb 2004 08:54:02 -0600
Lines: 24
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
Message-ID: <#DIRhzh#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
NNTP-Posting-Host: codemonkey.net 142.165.166.48
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP11.phx.gbl
Xref: cpmsftngxa06.phx.gbl
microsoft.public.dotnet.framework.aspnet:212973
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet

Yes, this is what I want to do (i.e., match clients with UserAgent==null)
j

Hi Jason,

Thank you for posting in community.
From your description, you're wondering how to set the browserCap'
filter
so as to capture those clients whose useragent is null, yes?
Regarding on the issue, I am
finding proper resource to assist you and we will update as soon as
posible.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security(This posting is provided "AS
IS",
with no warranties, and confers no rights.)
Click to expand...
Click to expand...
Click to expand...
 
Too bad.

Can you get this feedback to the Whidbey team, as there are a growing number
of popular crawlers that are simply using NULL as their UserAgent?

It would seem that some special attribute would have to be introduced to the
<case> tag to make this match.

Thanks,
j

Jim Cheshire said:
Jason,

That's just not going to be possible via the machine.config file. There is
no Regular Expression that will match a value that is not defined.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.

--------------------
From: "Jason Collins" <[email protected]>
References: <[email protected]>
Click to expand...
<[email protected]>
<#DIRhzh#[email protected]>
<CRta5im#[email protected]>
<l7L351y#[email protected]>
Subject: Re: <browserCaps> and NULL UserAgent
Date: Thu, 26 Feb 2004 16:23:54 -0600
Lines: 175
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
Message-ID: <[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
NNTP-Posting-Host: codemonkey.net 142.165.166.48
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP09.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:214088
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet

Because I'm want to do this centrally in the machine.config's
Click to expand...
so that it doesn't litter my code all over the place.

j


Eric Lawrence said:
Perhaps I'm missing something. Why not just test the
Request.Headers["User-Agent"] against null?

--
Thanks,

Eric Lawrence
Program Manager
Assistance and Worldwide Services

This posting is provided "AS IS" with no warranties, and confers no rights.

Hi Jason,

I have conferred with all of the experts, and the bottom line is that this
cannot be done. You can match an empty string, but you cannot match
something that is undefined. In order to match something with RegEx, you
must first define it.

In the process of troubleshooting your issue, I was pointed at a really
cool tool that I thought I'd share with you. Here it is:
Click to expand...
http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=C712
F
2DF-B026-4D58-8961-4EE2729D7322

Thanks.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.

--------------------
X-Tomcat-ID: 713102383
References: <[email protected]>
<[email protected]>
<#DIRhzh#[email protected]>
<CRta5im#[email protected]>
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
From: (e-mail address removed) (Jim Cheshire [MSFT])
Organization: Microsoft
Date: Tue, 24 Feb 2004 23:25:19 GMT
Subject: Re: <browserCaps> and NULL UserAgent
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet
Message-ID: <l7L351y#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
Lines: 75
NNTP-Posting-Host: TOMCATIMPORT1 10.201.218.122
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGXA06.phx.gbl!cpmsftngxa10.phx.gbl
Xref: cpmsftngxa06.phx.gbl
microsoft.public.dotnet.framework.aspnet:213409

Hi Jason,

I'm still researching this. It's going to be a bit longer before I'll
have
something for you.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.

--------------------
X-Tomcat-ID: 749098030
References: <[email protected]>
<[email protected]>
<#DIRhzh#[email protected]>
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
From: (e-mail address removed) (Jim Cheshire [MSFT])
Organization: Microsoft
Date: Mon, 23 Feb 2004 23:56:52 GMT
Subject: Re: <browserCaps> and NULL UserAgent
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet
Message-ID: <CRta5im#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
Lines: 47
Path: cpmsftngxa06.phx.gbl
Xref: cpmsftngxa06.phx.gbl
microsoft.public.dotnet.framework.aspnet:213135
NNTP-Posting-Host: tomcatimport2.phx.gbl 10.201.218.182

Hi Jason,

I am currently researching your issue and attempting to repro. I will
post
back here ASAP. It will be sometime tomorrow.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.


--------------------
From: "Jason Collins" <[email protected]>
References: <[email protected]>
<[email protected]>
Subject: Re: <browserCaps> and NULL UserAgent
Date: Mon, 23 Feb 2004 08:54:02 -0600
Lines: 24
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
Message-ID: <#DIRhzh#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
NNTP-Posting-Host: codemonkey.net 142.165.166.48
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP11.phx.gbl
Xref: cpmsftngxa06.phx.gbl
microsoft.public.dotnet.framework.aspnet:212973
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet

Yes, this is what I want to do (i.e., match clients with
UserAgent==null)
j

Hi Jason,

Thank you for posting in community.
From your description, you're wondering how to set the browserCap'
filter
so as to capture those clients whose useragent is null, yes?
Regarding on the issue, I am
finding proper resource to assist you and we will update as soon as
posible.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security(This posting is provided "AS
IS",
with no warranties, and confers no rights.)
Click to expand...
Click to expand...
Click to expand...
 
Jason,

It would take a little work on your part, but you could do this by simply
changing the default for "crawler" to true. You would then need to alter
each case statement to set crawler to false.

Jim Cheshire, MCSE, MCSD
ASP.NET
Developer Support
(e-mail address removed)
469-775-6577
Available from 9:00 CT - 6:00 CT, Monday - Friday

My Manager:
Jeremy Chapman
(e-mail address removed)
469-775-6360

Make sure that your ASP.NET applications are secure!
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnnetsec/ht
ml/secnetlpMSDN.asp


--------------------
From: "Jason Collins" <[email protected]>
References: <[email protected]>
Click to expand...
<[email protected]>
<#DIRhzh#[email protected]>
<CRta5im#[email protected]>
<l7L351y#[email protected]>
<[email protected]>
<#[email protected]>
Subject: Re: <browserCaps> and NULL UserAgent
Date: Fri, 27 Feb 2004 09:06:55 -0600
Lines: 244
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
Message-ID: <[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
NNTP-Posting-Host: codemonkey.net 142.165.166.48
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGXS01.phx.gbl!TK2MSFTNGXA05.phx.gbl!TK2MSFTNGP0
8.phx.gbl!TK2MSFTNGP10.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:214254
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet


Too bad.

Can you get this feedback to the Whidbey team, as there are a growing number
of popular crawlers that are simply using NULL as their UserAgent?

It would seem that some special attribute would have to be introduced to the
<case> tag to make this match.

Thanks,
j

Jim Cheshire said:
Jason,

That's just not going to be possible via the machine.config file. There is
no Regular Expression that will match a value that is not defined.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.

--------------------
From: "Jason Collins" <[email protected]>
References: <[email protected]>
Click to expand...
<[email protected]>
<#DIRhzh#[email protected]>
<CRta5im#[email protected]>
<l7L351y#[email protected]>
Subject: Re: <browserCaps> and NULL UserAgent
Date: Thu, 26 Feb 2004 16:23:54 -0600
Lines: 175
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
Message-ID: <[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
NNTP-Posting-Host: codemonkey.net 142.165.166.48
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP09.phx.gbl
Xref: cpmsftngxa06.phx.gbl microsoft.public.dotnet.framework.aspnet:214088
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet

Because I'm want to do this centrally in the machine.config's
Click to expand...
so that it doesn't litter my code all over the place.

j


Perhaps I'm missing something. Why not just test the
Request.Headers["User-Agent"] against null?

--
Thanks,

Eric Lawrence
Program Manager
Assistance and Worldwide Services

This posting is provided "AS IS" with no warranties, and confers no
rights.

Hi Jason,

I have conferred with all of the experts, and the bottom line is that
this
cannot be done. You can match an empty string, but you cannot match
something that is undefined. In order to match something with RegEx,
you
must first define it.

In the process of troubleshooting your issue, I was pointed at a really
cool tool that I thought I'd share with you. Here it is:
Click to expand...

http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=C71 2
F
2DF-B026-4D58-8961-4EE2729D7322

Thanks.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.

--------------------
X-Tomcat-ID: 713102383
References: <[email protected]>
<[email protected]>
<#DIRhzh#[email protected]>
<CRta5im#[email protected]>
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
From: (e-mail address removed) (Jim Cheshire [MSFT])
Organization: Microsoft
Date: Tue, 24 Feb 2004 23:25:19 GMT
Subject: Re: <browserCaps> and NULL UserAgent
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet
Message-ID: <l7L351y#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
Lines: 75
NNTP-Posting-Host: TOMCATIMPORT1 10.201.218.122
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGXA06.phx.gbl!cpmsftngxa10.phx.gbl
Xref: cpmsftngxa06.phx.gbl
microsoft.public.dotnet.framework.aspnet:213409

Hi Jason,

I'm still researching this. It's going to be a bit longer before I'll
have
something for you.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.

--------------------
X-Tomcat-ID: 749098030
References: <[email protected]>
<[email protected]>
<#DIRhzh#[email protected]>
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
From: (e-mail address removed) (Jim Cheshire [MSFT])
Organization: Microsoft
Date: Mon, 23 Feb 2004 23:56:52 GMT
Subject: Re: <browserCaps> and NULL UserAgent
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet
Message-ID: <CRta5im#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
Lines: 47
Path: cpmsftngxa06.phx.gbl
Xref: cpmsftngxa06.phx.gbl
microsoft.public.dotnet.framework.aspnet:213135
NNTP-Posting-Host: tomcatimport2.phx.gbl 10.201.218.182

Hi Jason,

I am currently researching your issue and attempting to repro. I will
post
back here ASAP. It will be sometime tomorrow.

Jim Cheshire, MCSE, MCSD [MSFT]
ASP.NET
Developer Support
(e-mail address removed)

This post is provided as-is with no warranties and confers no rights.


--------------------
From: "Jason Collins" <[email protected]>
References: <[email protected]>
<[email protected]>
Subject: Re: <browserCaps> and NULL UserAgent
Date: Mon, 23 Feb 2004 08:54:02 -0600
Lines: 24
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2800.1158
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
Message-ID: <#DIRhzh#[email protected]>
Newsgroups: microsoft.public.dotnet.framework.aspnet
NNTP-Posting-Host: codemonkey.net 142.165.166.48
Path: cpmsftngxa06.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGP11.phx.gbl
Xref: cpmsftngxa06.phx.gbl
microsoft.public.dotnet.framework.aspnet:212973
X-Tomcat-NG: microsoft.public.dotnet.framework.aspnet

Yes, this is what I want to do (i.e., match clients with
UserAgent==null)
j

Hi Jason,

Thank you for posting in community.
From your description, you're wondering how to set the browserCap'
filter
so as to capture those clients whose useragent is null, yes?
Regarding on the issue, I am
finding proper resource to assist you and we will update as
Click to expand...
soon
as
posible.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security(This posting is provided "AS
IS",
with no warranties, and confers no rights.)
Click to expand...
Click to expand...
Click to expand...
 
Back
Top