Email - How to find out how many new Emails in POP server

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

Hi guys,

I am making a small program to retrieve e-mails from POP accounts. I got all
the e-mail parsing stuff figured out, but I cannot seem to come up with a way
to find out which e-mails are NEW so I don't have to retrieve them all. If
you have experience with this kind of thing, you know that the server creates
unique IDs for all the messages, but this IDs are not guaranteed to be
unique, since they can be reused once a message is deleted.
I wonder how Outlook checks for new messages... it happens very fast which
would indicate it is not really checking EVERY ID on its DB against the POP
server...

Any ideas?

Thank you in advance!

Juan Romero
 
The simplest thing I would do is create an index of all the mail in the
user's mailbox.

Structure MailItem
FileName as string
IsRead as boolean
IsReplyTo as boolean
End Structure

Create a hashtable, store mailitem in it, used the message guid as the key.

Serialize the hashtable.

At the server level,
deserialize mailbox index, update index, serialize for each

Mail read, delete, new message

Or you could store the index in a database (Access, MSSQL, Oracle, Excel)

The pop server is doing this on every action for a message, and believe it
or not, its pretty fast. The question is how efficient can you make it. The
most efficient would most likely be SQL or Oracle. Second would be Access.
Finally, Serialization.
 
=?Utf-8?B?TWFkZXN0cm8=?= said:
I am making a small program to retrieve e-mails from POP accounts. I
got all the e-mail parsing stuff figured out, but I cannot seem to
come up with a way to find out which e-mails are NEW so I don't have
to retrieve them all. If you have experience with this kind of thing,

POP3 does not keep track of what is new or not, you have to do that. You can use the UIDL command
to do this. There is a very basic demo here of POP3 client (Does not show UIDL, but it supports
UIDL):
http://www.codeproject.com/useritems/POP3Client.asp


--
Chad Z. Hower (a.k.a. Kudzu) - http://www.hower.org/Kudzu/
"Programming is an art form that fights back"

Get your ASP.NET in gear with IntraWeb!
http://www.atozed.com/IntraWeb/
 
Your sample code only reflects client activity and does not emphsize UIDL or
how you would implement such a concept, nor does it say why you would want to.

The question was directed as to what approach one would take to implement
the monitoring of new/read messages at the server level. Many popular POP
servers support this concept now, and is easily proven by using common
clients on a couple of pc's. Each client will know which messages, the
previous client has read and which messages are still unread. This indicates
that the server is caching that information and presenting it to the client
requesting a LIST.

The RFC for POP version 3 states that POP is designed to house new messages
on the server until a client downloads them and ultimately deletes them. It
was not designed for enhanced mail manipulation. If you want to be elaborate
with message management, the RFC points you to IMAP.

How ever, since the RFC supports flagging a message for deletion, we can
take the opportuinity to extend the bit into a byte and use it as a tripple
state flag {New, Read, Delete}

Here is a bit of skeleton code to help illustrate my point. I hope it
helps. I realize that it is far from complete and less than optimal, but I
am sure you can glean from it, what I was trying to demonstrate and move
forward with your project.

<Serializable()> Public Enum MessageStates As Byte
[new] = 0
[read] = 1
[delete] = 2
End Enum

<Serializable()> Public Class Mailbox

Private m_MailMessages As MailMessages

Public Sub New()
m_MailMessages = New MailMessages
End Sub

Public Property Messages() As MailMessages
Get
Return m_MailMessages
End Get
Set(ByVal Value As MailMessages)
m_MailMessages = Value
End Set
End Property

Public ReadOnly Property Count() As Integer
Get

Dim TheCount As Integer

For Each mm As MailMessage In m_MailMessages
If mm.MessageSate <> MessageStates.delete Then
TheCount = TheCount + 1
End If
Next

Return TheCount

End Get

End Property

Public NotInheritable Class MailMessages
Inherits Hashtable

Public Sub New()
MyBase.New()
End Sub

Public Overloads Sub Add(ByVal MessageID As String, ByVal MessagePtr As
MailMessage)
MyBase.Add(MessageID, MessagePtr)
End Sub

Public Overloads Function ContainsKey(ByVal MessageID As String) As
Boolean

If MyBase.ContainsKey(MessageID) = True Then
If CType(MyBase.Item(MessageID), MailMessage).MessageSate <>
MessageStates.delete Then
Return True
Else
Return False
End If
End If

End Function

Public Overrides Function GetEnumerator() As
System.Collections.IDictionaryEnumerator
Return New MailMessageEnumerator(Me)
End Function

Public Overloads Sub Remove(ByVal MessageID As String)

If MyBase.ContainsKey(MessageID) = True Then
CType(MyBase.Item(MessageID), MailMessage).MessageSate =
MessageStates.delete
End If

End Sub

Default Public Overloads Property Item(ByVal MessageID As String) As
MailMessage
Get
Return CType(MyBase.Item(MessageID), MailMessage)
End Get
Set(ByVal Value As MailMessage)
MyBase.Item(MessageID) = Value
End Set
End Property
End Class

Public NotInheritable Class MailMessageEnumerator
Implements IEnumerator

Private m_Enumerable As IDictionaryEnumerator

Public Sub New(ByVal MessageTable As MailMessages)
m_Enumerable = MessageTable.GetEnumerator
End Sub

Private ReadOnly Property IEnumerator_Current() As Object Implements
System.Collections.IEnumerator.Current
Get
Return m_Enumerable.Current
End Get
End Property

Private Function IEnumerator_MoveNext() As Boolean Implements
System.Collections.IEnumerator.MoveNext
m_Enumerable.MoveNext()
End Function

Private Sub IEnumerator_Reset() Implements
System.Collections.IEnumerator.Reset
m_Enumerable.Reset()
End Sub

Public ReadOnly Property Current() As MailMessage
Get
Return CType(IEnumerator_Current, MailMessage)
End Get
End Property

Public Function MoveNext() As Boolean
m_Enumerable.MoveNext()
End Function

Public Sub Reset()
IEnumerator_Reset()
End Sub

End Class

Public NotInheritable Class MailMessage
Private m_MessageID As String
Private m_MessageState As MessageStates

Public Sub New(ByVal MessageID As String)
m_MessageID = MessageID
m_MessageState = MessageStates.[new]
End Sub

Public ReadOnly Property MessageID() As String
Get
Return m_MessageID
End Get
End Property

Public Property MessageSate() As MessageStates
Get
Return m_MessageState
End Get
Set(ByVal Value As MessageStates)
m_MessageState = Value
End Set
End Property

End Class

End Class

Public Class POPServer

Private m_Shutdown As Boolean
Private m_LoggedInUsers As Hashtable
Private m_Sessions As Hashtable
Private m_Mailboxes As Hashtable

Private Structure SessionInfo
Dim ClientSession As POPSession
Dim UserName As String
End Structure

Public Sub New()
m_Shutdown = False
m_Sessions = New Hashtable
m_Mailboxes = New Hashtable
m_LoggedInUsers = New Hashtable
End Sub

'main loop
Sub MainLoop()

Dim poplistener As System.Net.Sockets.TcpListener
poplistener = New
System.Net.Sockets.TcpListener(System.Net.IPAddress.Parse("127.0.0.1"), 5555)

'With a TCPLisener, wait for a new connection
poplistener.Start()

Do While Not m_Shutdown

'Accept any new clients out there
Dim POPClient As System.Net.Sockets.TcpClient

POPClient = poplistener.AcceptTcpClient

'We have a connection

'Pass it on
Dim ThisSessionInfo As SessionInfo
Dim ThisSession As POPSession
Dim POPThread As System.Threading.Thread
Dim SessionID As String

SessionID = System.Guid.NewGuid.ToString

ThisSession = New POPSession(Me, POPClient, SessionID)

With ThisSessionInfo
.ClientSession = ThisSession
End With

m_Sessions.Add(SessionID, ThisSessionInfo)

POPThread = New System.Threading.Thread(AddressOf
ThisSession.SessionStart)

POPThread.Start()

Loop

poplistener.Stop()

End Sub

Public Function SessionLogin(ByVal SessionID As String, ByVal UserName As
String, ByVal Password As String) As Byte()

Dim bSuccess As Boolean

'Login Validation logic here

If bSuccess Then

If Not m_LoggedInUsers.ContainsKey(UserName.ToUpper) Then
Dim SessionMap As Hashtable
SessionMap = New Hashtable

SessionMap.Add(SessionID, SessionID)

m_LoggedInUsers.Add(UserName, SessionMap)

Else

Dim SessionMap As Hashtable

SessionMap = CType(m_LoggedInUsers(UserName), Hashtable)
SessionMap.Add(SessionID, SessionID)

End If

If m_Sessions.ContainsKey(SessionID) Then
Dim ThisSessionInfo As SessionInfo
ThisSessionInfo = CType(m_Sessions.Item(SessionID), SessionInfo)
ThisSessionInfo.UserName = UserName.ToUpper
End If

Dim ThisMailbox As Mailbox

'Deserialize the mailbox

m_Mailboxes.Add(UserName, ThisMailbox)

End If

End Function

Public Function SessionRETR(ByVal SessionID As String, ByVal MessageID As
String) As Byte()

Dim bData() As Byte
Dim bSuccess As Boolean

'Validate our session

'Read the message into our byte array

If bSuccess Then
Dim ThisSessionInfo As SessionInfo

If m_Sessions.ContainsKey(SessionID) Then

ThisSessionInfo = m_Sessions.Item(SessionID)

If m_Mailboxes.ContainsKey(ThisSessionInfo.UserName) Then

Dim ThisMailbox As Mailbox

ThisMailbox = CType(m_Mailboxes.Item(ThisSessionInfo.UserName),
Mailbox)
ThisMailbox.Messages(MessageID).MessageSate = MessageStates.read

Return bData

End If

End If

End If


End Function

Public Sub SessionLogout(ByVal SessionID As String)

Dim ThisSessionInfo As SessionInfo

'Determine if we need to close this mailbox
If m_Sessions.ContainsKey(SessionID) Then

ThisSessionInfo = m_Sessions.Item(SessionID)

If m_LoggedInUsers.ContainsKey(ThisSessionInfo.UserName) Then

Dim SessionMap As Hashtable

SessionMap = m_LoggedInUsers.Item(ThisSessionInfo.UserName)

If SessionMap.ContainsKey(SessionID) Then

SessionMap.Remove(SessionMap.Item(SessionID))

If SessionMap.Count = 0 Then

If m_Mailboxes.ContainsKey(ThisSessionInfo.UserName) Then
Dim ThisMailBox As Mailbox
ThisMailBox =
CType(m_Mailboxes.Item(ThisSessionInfo.UserName), Mailbox)

'Check for deleted messages

SyncLock Me

For Each mm As Mailbox.MailMessage In ThisMailBox.Messages
If mm.MessageSate = MessageStates.delete Then
'Delete the message
ThisMailBox.Messages.Remove(mm)
End If
Next

End SyncLock

'Serialize the mailbox back to the hard drive

End If

m_Mailboxes.Remove(ThisSessionInfo.UserName)

m_LoggedInUsers.Remove(m_LoggedInUsers.Item(ThisSessionInfo.UserName))

End If

End If

End If

End If

End Sub

End Class

Public Class POPSession

'I think I should use delegate, but I can never figure them out

'Public Delegate Function Login(ByVal SessionID As String, ByVal UserName
As String, ByVal Password As String) As Byte()
'Public Delegate Function RETR(ByVal SessionID As String, ByVal MessageID
As String) As Byte()
'Public Delegate Sub Logout(ByVal SessionID As String)

'Public DoLogin As Login
'Public DoRETR As RETR
'Public DoLogout As Logout

Private m_PopServer As POPServer
Private m_TcpClient As System.Net.Sockets.TcpClient
Private m_SessionID As String
Private m_UserName As String

Public Sub New(ByVal PopServer As POPServer, ByVal POPClient As
System.Net.Sockets.TcpClient, ByVal SessionID As String)
m_PopServer = PopServer
m_TcpClient = POPClient
m_SessionID = SessionID
End Sub

Public Sub SessionStart()

Dim clientcommand As String
Dim commandargs() As String

'Listen for new commands
While 1 = 1

'parsing logic here

Select Case clientcommand.ToUpper
Case "PASS"
'Assuming we already had the USER command
LocalPASS(commandargs(0))
Case "RETR"
LocalRETR(commandargs(0))
Case "QUIT"
LocalLogout()
End Select

End While

End Sub

Private Sub LocalPASS(ByVal Password As String)

Dim bData As Byte()

bData = m_PopServer.SessionLogin(m_SessionID, m_UserName, Password)

sendData(bData)

End Sub

Private Sub LocalRETR(ByVal MessageID As String)
Dim bData As Byte()

bData = m_PopServer.SessionRETR(m_SessionID, MessageID)

sendData(bData)

End Sub

Private Sub LocalLogout()
m_PopServer.SessionLogout(m_SessionID)
End Sub

Private Sub sendData(ByVal DataToSend As Byte())
Dim stream As System.net.Sockets.NetworkStream = m_TcpClient.GetStream()
'Log Activity here

'Send Data
stream.Write(DataToSend, 0, DataToSend.Length)
End Sub

End Class
 
Nevermind, it was too late last night. Madestro was asking about the client.

AMDIRT said:
Your sample code only reflects client activity and does not emphsize UIDL or
how you would implement such a concept, nor does it say why you would want to.

The question was directed as to what approach one would take to implement
the monitoring of new/read messages at the server level. Many popular POP
servers support this concept now, and is easily proven by using common
clients on a couple of pc's. Each client will know which messages, the
previous client has read and which messages are still unread. This indicates
that the server is caching that information and presenting it to the client
requesting a LIST.

The RFC for POP version 3 states that POP is designed to house new messages
on the server until a client downloads them and ultimately deletes them. It
was not designed for enhanced mail manipulation. If you want to be elaborate
with message management, the RFC points you to IMAP.

How ever, since the RFC supports flagging a message for deletion, we can
take the opportuinity to extend the bit into a byte and use it as a tripple
state flag {New, Read, Delete}

Here is a bit of skeleton code to help illustrate my point. I hope it
helps. I realize that it is far from complete and less than optimal, but I
am sure you can glean from it, what I was trying to demonstrate and move
forward with your project.

<Serializable()> Public Enum MessageStates As Byte
[new] = 0
[read] = 1
[delete] = 2
End Enum

<Serializable()> Public Class Mailbox

Private m_MailMessages As MailMessages

Public Sub New()
m_MailMessages = New MailMessages
End Sub

Public Property Messages() As MailMessages
Get
Return m_MailMessages
End Get
Set(ByVal Value As MailMessages)
m_MailMessages = Value
End Set
End Property

Public ReadOnly Property Count() As Integer
Get

Dim TheCount As Integer

For Each mm As MailMessage In m_MailMessages
If mm.MessageSate <> MessageStates.delete Then
TheCount = TheCount + 1
End If
Next

Return TheCount

End Get

End Property

Public NotInheritable Class MailMessages
Inherits Hashtable

Public Sub New()
MyBase.New()
End Sub

Public Overloads Sub Add(ByVal MessageID As String, ByVal MessagePtr As
MailMessage)
MyBase.Add(MessageID, MessagePtr)
End Sub

Public Overloads Function ContainsKey(ByVal MessageID As String) As
Boolean

If MyBase.ContainsKey(MessageID) = True Then
If CType(MyBase.Item(MessageID), MailMessage).MessageSate <>
MessageStates.delete Then
Return True
Else
Return False
End If
End If

End Function

Public Overrides Function GetEnumerator() As
System.Collections.IDictionaryEnumerator
Return New MailMessageEnumerator(Me)
End Function

Public Overloads Sub Remove(ByVal MessageID As String)

If MyBase.ContainsKey(MessageID) = True Then
CType(MyBase.Item(MessageID), MailMessage).MessageSate =
MessageStates.delete
End If

End Sub

Default Public Overloads Property Item(ByVal MessageID As String) As
MailMessage
Get
Return CType(MyBase.Item(MessageID), MailMessage)
End Get
Set(ByVal Value As MailMessage)
MyBase.Item(MessageID) = Value
End Set
End Property
End Class

Public NotInheritable Class MailMessageEnumerator
Implements IEnumerator

Private m_Enumerable As IDictionaryEnumerator

Public Sub New(ByVal MessageTable As MailMessages)
m_Enumerable = MessageTable.GetEnumerator
End Sub

Private ReadOnly Property IEnumerator_Current() As Object Implements
System.Collections.IEnumerator.Current
Get
Return m_Enumerable.Current
End Get
End Property

Private Function IEnumerator_MoveNext() As Boolean Implements
System.Collections.IEnumerator.MoveNext
m_Enumerable.MoveNext()
End Function

Private Sub IEnumerator_Reset() Implements
System.Collections.IEnumerator.Reset
m_Enumerable.Reset()
End Sub

Public ReadOnly Property Current() As MailMessage
Get
Return CType(IEnumerator_Current, MailMessage)
End Get
End Property

Public Function MoveNext() As Boolean
m_Enumerable.MoveNext()
End Function

Public Sub Reset()
IEnumerator_Reset()
End Sub

End Class

Public NotInheritable Class MailMessage
Private m_MessageID As String
Private m_MessageState As MessageStates

Public Sub New(ByVal MessageID As String)
m_MessageID = MessageID
m_MessageState = MessageStates.[new]
End Sub

Public ReadOnly Property MessageID() As String
Get
Return m_MessageID
End Get
End Property

Public Property MessageSate() As MessageStates
Get
Return m_MessageState
End Get
Set(ByVal Value As MessageStates)
m_MessageState = Value
End Set
End Property

End Class

End Class

Public Class POPServer

Private m_Shutdown As Boolean
Private m_LoggedInUsers As Hashtable
Private m_Sessions As Hashtable
Private m_Mailboxes As Hashtable

Private Structure SessionInfo
Dim ClientSession As POPSession
Dim UserName As String
End Structure

Public Sub New()
m_Shutdown = False
m_Sessions = New Hashtable
m_Mailboxes = New Hashtable
m_LoggedInUsers = New Hashtable
End Sub

'main loop
Sub MainLoop()

Dim poplistener As System.Net.Sockets.TcpListener
poplistener = New
System.Net.Sockets.TcpListener(System.Net.IPAddress.Parse("127.0.0.1"), 5555)

'With a TCPLisener, wait for a new connection
poplistener.Start()

Do While Not m_Shutdown

'Accept any new clients out there
Dim POPClient As System.Net.Sockets.TcpClient

POPClient = poplistener.AcceptTcpClient

'We have a connection

'Pass it on
Dim ThisSessionInfo As SessionInfo
Dim ThisSession As POPSession
Dim POPThread As System.Threading.Thread
Dim SessionID As String

SessionID = System.Guid.NewGuid.ToString

ThisSession = New POPSession(Me, POPClient, SessionID)

With ThisSessionInfo
.ClientSession = ThisSession
End With

m_Sessions.Add(SessionID, ThisSessionInfo)

POPThread = New System.Threading.Thread(AddressOf
ThisSession.SessionStart)

POPThread.Start()

Loop

poplistener.Stop()

End Sub

Public Function SessionLogin(ByVal SessionID As String, ByVal UserName As
String, ByVal Password As String) As Byte()

Dim bSuccess As Boolean

'Login Validation logic here

If bSuccess Then

If Not m_LoggedInUsers.ContainsKey(UserName.ToUpper) Then
Dim SessionMap As Hashtable
SessionMap = New Hashtable

SessionMap.Add(SessionID, SessionID)

m_LoggedInUsers.Add(UserName, SessionMap)

Else

Dim SessionMap As Hashtable

SessionMap = CType(m_LoggedInUsers(UserName), Hashtable)
SessionMap.Add(SessionID, SessionID)

End If

If m_Sessions.ContainsKey(SessionID) Then
Dim ThisSessionInfo As SessionInfo
ThisSessionInfo = CType(m_Sessions.Item(SessionID), SessionInfo)
ThisSessionInfo.UserName = UserName.ToUpper
End If

Dim ThisMailbox As Mailbox

'Deserialize the mailbox

m_Mailboxes.Add(UserName, ThisMailbox)

End If

End Function

Public Function SessionRETR(ByVal SessionID As String, ByVal MessageID As
String) As Byte()

Dim bData() As Byte
Dim bSuccess As Boolean

'Validate our session
 
Back
Top