Exchange Event 1025 - SLINK::EcUpdate Error: 0x8004010f

Background:

Event 1025 is logged randomly in the application event log.

As per MS this event is a generic warning that can occur for various reasons that may include logical corruption in the database. In addition this event may occur repeatedly with different error codes.

In my experience this event is logged because there are corrupted items in Public Folders or Mailboxes.

==============================================================
Event Type: Warning
Event Source: MSExchangeIS Mailbox Store
Event Category: General
Event ID: 1025
Date:  7/05/2012
Time:  8:30:20 AM
User:  N/A
Computer: <>
Description:
An error occurred on database "<>".
 Function name or description of problem: SLINK::EcUpdate
Error: 0x8004010f
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 46 49 44 3a 20 36 2d 46   FID: 6-F
0008: 37 42 36 39 2c 20 53 46   7B69, SF
0010: 49 44 3a 20 36 2d 38 43   ID: 6-8C
0018: 35 42 46 34 46            5BF4F  
==============================================================

Diagnosis

Start PFDAVAdmin and connect to the mailbox server reporting the 1025 event.Select
All Mailboxes in the radial button.

Tools Menu, choose Export Properties. Choose ‘All Folders’. Select only the following properties to export:

PR_DISPLAY_NAME : 0x3001001E
PR_DIAPLY_TYPE : 0×39000003
ptagFID : 0×67480014

Select an Output file and click OK.

Now go back to the 1025 event and look at the Data section of the event, make sure Data type is type
Bytes, look at the entry that says FID and note the FID value. Now go back to the Output file and
do a search on this FID value.

You will find the mailbox\folder\item has this FID value, this item is corrupted and needs to be deleted, the best way to delete this corrupted item is via a mailbox move, move the mailbox with
corrupted items to different database on the same or different mailbox servers, make sure skip
corrupted items option is checked during the move.

If the FID value is not present in the Output file, then repeat the above procedure on public folders.

Automate Public Folder Permissions Backup via PFDAVAdmin

PFDAVAdmin has been around for sometime now but its command line options are not well known and nor well documented.

Navigate to the PFDAVAdmin executable from the command shell and execute PFDAVAdmin /?, Look closely and you will notice some caveats, the export
example does not list the -f option (listed in usage) and the typo in scope TopLevePF.

Example1: Backup permissions of Public Folder\Test\abc

Public Folder Server Name - MBX1
Username - pf_service
Password - @#WER56
Domain - XYZ

PFDAVAdmin -export -permissions ntaccount -s MBX1 -scope "Public Folders\Test\abc" -user pf_service -password @#WER56 -domain XYZ -f C:\PFDAVAdmin\abc.txt

Note that the domain switch is necessary, domain\user format does not
parse well with pfdavadmin.

Now lets automate the above command via windows scheduled task, quick note
on the account via which the scheduled task is executed, this account requires execute rights on c:\windows\system32\cmd.exe and should be a memeber of the local administrators group, If not then the scheduled task will error out,another workaround is domain admins membership to this service account, depending on your organizations security policies this may not be available.

to be contd..

Public Folder Recovery

Problem

A badly scoped powershell to remove a exchange server as PF replica deletes multiple public folders..

Solution

PFDAVAdmin was able to recover the Public Folders which were smaller in size but we had a PF which was in the range of 380gigs, PFDAVAdmin hanged while recovering and timed-out,The following events were logged in Application log,

The PF database locked for a few mins and none of the PF's on this folder
were available, this issue auto resolved and the PF's were available after 5-6 mins.

PFDAVAdmin was run again from a different system (AD server),again it hanged but there were no time-outs reported in the Application Log, Over a period of time some PF's were recovered but not in the original hierarchy but at the root of the PF tree.

Note that the custom proxy addresses are gone once you recover the PF's via
PFDAVAdmin or via a backup restore.

Since the 380 gig PF was not recoverable via PFDAVAdmin we used Ontrack Power Controls. The PF database backup taken prior to this issue was
restored to a temp drive and then a PF-by-PF restoration executed, permissions were not lost though custom proxy addresses had to be manually
populated.

Recommendation

If you have a massive Public Folder infra then make sure to backup the permissions and proxy addresses atleast once every week, PFDAVAdmin command
line options can be used to automate this procedure.
ADModify could also be used, but this requires the proxies to be first
exported (removed) and stored in a xml file and then imported back.

PFDAVAdmin doesnot work on such massive PF's and restore from backup plus a tool like Power Controls is the only available option.Also as mentioned above make sure to backup proxy addresses and permissions weekly.

to be contd..