Tuesday, July 5, 2011

Backup and Restore in SharePoint 2010

Problem

So what are the 'out of the box' Backup and Restore options available in SharePoint 2010? How do we backup the SharePoint farm, web applications and different components? How have Backup and Restore processes have improved from previous versions of SharePoint to SharePoint 2010?

Solution

As a SharePoint Administrator, you need to set up a backup plan so that you can restore your SharePoint farm or SharePoint web applications without any hassles if a disaster happens or even to create a replica of your production environment for QA and development purposes.

Backup and restore functionalities in SharePoint 2010 have matured tremendously from previous versions. Now you can backup and restore farms as well as web applications from the Central Administration more easily. You can even take granular backups. For example, you can take a backup of any specific site collection from the Central Administration. You can export site content to the file system and import it into another site.

And yes. now you don't need to setup another temporary farm to just restore some content from the backup, you can simply restore a content database on any SQL Server instance and then use the unattached content database recovery feature from Central Administration to restore the selected content.

This tip discusses the Backup and Restore of a SharePoint farm, web applications and components from the Central Administration as well as from PowerShell and the STSADM command.

Backing-up SharePoint farm from the Central Administration

Go to START -> All Programs -> Microsoft SharePoint 2010 Products -> SharePoint 2010 Central Administration.

On the left side, click on the Central Administration link and in right side, in the detail pane, you will see Backup and Restore options as shown below.

You can also directly click on the Backup and Restore link in the left side to go to Backup and Restore page as shown next.



On the Backup and Restore page, you can see two sections, Farm Backup and Restore which allows you to take backup of SharePoint farm and web applications and also restore them when required. You can even see the status of any running backup and restore operations and also view the history of any backups taken so far on the specified location. (I will be talking about Granular Backup in more detail in a future tip).

Click on the Perform a backup link under Farm Backup and Restore to start the backup operation. You need to make sure SharePoint 2010 Timer (used to send notifications and perform scheduled tasks for SharePoint) and SharePoint 2010 Administration (used to perform administrative tasks for SharePoint) windows services are running or else you will not be able to perform backup and restore operations.

On the first step, you can select either to backup a whole farm or an individual web applications or components from it. In this example we will take a backup of a whole SharePoint farm. After selecting "Farm" we see all the components within it by default get selected.

On the second step, you can re-define the components you are going to take a backup of. Next you need to define backup types. i.e. full backup or differential backup (which you can take only if you have taken at least one full backup beforehand). A differential backup combines the changes which happened since the last full backup. If your content database is not that huge, it is recommended to setup for full backup only which would be easier during a restoration. If you have a very large database, then you can combine these two backup types for reduced backup time and less storage. With restoration, you need to restore the latest full backup first followed by the latest differential backup.

By default both configuration settings and content databases are backed up as part of backup operation, though if required you can choose to backup only the farm configuration. And finally you need to specify a backup location where a folder will be created and backed up components will be stored. On the bottom section of the next screen you can see the estimated disk space required for the selected backup type.

Click on the Start Backup button to create a timer service job which will run in the background and take this backup. As I said before you need to have both the SharePoint 2010 Timer and SharePoint 2010 Administration windows services are running.

On the Backup and Restore Job Status page, you can monitor the status of your backup or restore job (pictured below). In this case, 43 items have been backed up out of total 192. This page gets refreshed every 30 seconds to reflect current information but you can also refresh it by clicking on Refresh link.

The Default Backup and Restore Settings page allows you to specify the number of threads which will be used during a backup operation and the number of threads which will be used during a restore operation. By default it is 3 threads -- if the number of threads were higher, performance would be higher but it would consume more resources. Also going through the logs generated by these higher number of threads would be a little difficult.

Next you can specify the default backup location, each timer job will create a new sub folder and store the content within it.

Backing-up web-application(s) from the Central Administration

The process of taking backup of web-applications is not much different from taking backup of the entire farm. In the first step while selecting components to backup, instead of selecting farm, you need to select individual web applications or components for which you want to take backup of.

As you had options while taking a backup of the farm, these options are available here as well. You can choose the component if you need to do so and you can specify whether you want to take full back or differential backup and then you can specify the location where you want backup to be stored.

On the Backup and Restore Job Status page you can see and monitor the status of your backup or restore operations.

Restoring SharePoint farm/web application(s) from the Central Administration

To restore a SharePoint farm and web applications, go to Backup and Restore page as shown below. Click on theRestore from a backup link to go to restore page.

On the first step of restore, specify the shared location where you have stored the backup and click on the Refresh button. It will display a list of all the backups available on the specified location. Choose the backup that you want to restore and click on the Next button. As with backup, you need to ensure both the SharePoint 2010 Timer and SharePoint 2010 Administration windows services are running.

On the second step of the restore, all the components that the selected backup contains will be displayed as shown below. You can decide which of the components you want to restore from the selected backup. For example you can see below I have selected to restore the AdventureWorks web application.

On the third and final step of the restore, you can re-purpose your selection of components to restore. Next you need to specify that the restore will use the same configuration, web application name, database server name, database name or you want to ignore the existing configuration and use a new configuration for the restore. This gives you flexibility if you are restoring the same backup to a different environment.

On the Backup and Restore Job Status page, you can monitor the restore operation progress. Though the restore status page gets refreshed in every 30 seconds, you can click on Refresh link to refresh the status of the restore operation on-demand. As I said above, by default SharePoint uses 3 restore threads during restore operation which you can see below. You can change this default settings by going to Default Backup and Restore settings page.

Backing-up farm or web-application(s) using PowerShell and STSADM tool

Central Administration is not the only way to do backup and restore in SharePoint 2010. SharePoint 2010 provides several PowerShell cmdlets to perform these operations, in other words you can script these operations and schedule it with Windows Task Scheduler to make it run on scheduled basis.

Backup-SPFarm

The Backup-SPFarm command is used to backup the entire farm, web applications or a farm's components or farm configuration, etc. The general syntax of this command is:

Backup-SPFarm -Directory -BackupMethod [Full | Differential] -Item [-Verbose]

To launch the SharePoint 2010 PowerShell console, go to START -> All Programs -> Microsoft SharePoint 2010 Products -> SharePoint 2010 Management Shell and click on it. To take a backup of an entire farm, we need to use the Backup-SPFarm command and specify the location where a backup will be stored and the type of backup we want to take (shown below). A verbose parameter gives details about the status during backup:

To backup any individual content database or web application, use the Item parameter (as shown below). The rest other parameters remain same:

Backup-SPFarm -Directory \\MKTARALIW2K8R2\SharePointBackupFolder -Item "Farm\Microsoft SharePoint Foundation Web Application\AdventureWorks" -BackupMethod Full -verbose

Restoring a farm or web-application(s) using PowerShell and STSADM tool

To restore an entire farm use the Restore-SPFarm command and specify the location where backup is stored, restore method to overwrite existing deployment or whether restoring afresh on new environment:

Restore-SPFarm -Directory \\MKTARALIW2K8R2\SharePointBackupFolder -RestoreMethod Overwrite

You can specify a specific component to restore from the existing farm backup using the Item parameter, for example I am restoring only the AdventureWorks web application from the available farm backup as shown below:

Restore-SPFarm -Directory \\MKTARALIW2K8R2\SharePointBackupFolder -Item "Farm\Microsoft SharePoint Foundation Web Application\AdventureWorks" -RestoreMethod Overwrite

Note: STSADM is an administrative tool which has been in previous versions of SharePoint for all the administrative tasks and still supported (though enhanced in some ways) in SharePoint 2010. However, STSADM has been deprecated as well. Instead of using this tool, its recommended to use PowerShell cmdlets. For that reason I will not be talking about it in details.

Notes

  • Please note though you can still use the old STSADM tool to perform backup and restore but it is recommended not to use this tool (unless you are forced to use it for backward compatibility) as this tool has been deprecated and provided in SharePoint 2010 only for backward compatibility.
  • The spbrtoc.xml file on the backup/restore location keeps the backup and restore history. The spbackup.xml file in the backup/restore folder keeps the status about each component involved in the operation. In case of any error you can refer to Spbackup.log file in the specified location for more detail.
  • When you take a farm backup, certificates are not backed up with it so you need to plan for its backup separately. Likewise if you use forms-based authentication you need to ensure you take backup of manually updated web.config separately (if the web.config gets updated by Central Administration, there is no need for this additional step).
  • To improve the performance of Backup and Restore operations, make sure you use the local path location to minimize the latency and then move the file to network share if required. For example, if you have both SharePoint site and SQL Server on the same machine, use a local path. If not then use a network shared location which can be accessed quickly (probably keeping the shared location on the SQL box or keeping it on fast drives).
  • Only a member of the Farm Administrators group can take a backup using Central Administration. Also you need to have Full Control on the shared given to the SQL Server service's account, SharePoint 2010 Timer Service and Central Administration's application pool identity account.
  • Additional related PowerShell cmdlets are:
    • Backup-SPConfigurationDatabase which is used take a 'configuration only' backup of the current farm or a 'configuration only' backup of a separate configuration database which is not attached to the current farm.
    • Get-SPBackupHistory is used display the history of backup and restore operations run so far on the specified location.

Getting Started with the SharePoint Logging and Reporting Database in SharePoint 2010

Problem

In my last tip I talked about diagnostic logging in SharePoint 2010 and how it helps in troubleshooting. Troubleshooting with trace files is good but wouldn't it be cool if we have all the logging (usage and health) information on single place in consolidated form which help you to build reports and do analysis on collected data from all the servers of the farm? I heard SharePoint 2010 has new logging database. What is it, what does it store? How we can control the amount of information to be logged in the SharePoint logging database?

Solution

SharePoint 2010 is now capable of accumulating all the usage and health related data (log files) from all servers in the farm into a new SQL Server database. This centralized consolidation of data, from all servers helps you to better generate reports for analysis purposes and troubleshooting. You can even use it as external content types or a list to Business Connectivity Services(BCS). By default this is enabled and data gets aggregated using a timer job.

As opposed to other SharePoint databases, this database is fully supported for reading and writing whenever you need to do so, as the schema is public.

Now if you are wondering why is only this database supported for reading and writing then the reason is very simple. The logging database aggregates usage and health data from trace files from all servers (already gathered information on the servers).

Before digging more into this database, lets see how to configure and control what and when to be aggregated in the logging database. Go to the Monitoring link on the left side of SharePoint 2010 Central Administration and click on the "Configure usage and health data collection" link under Reporting as shown below:

As I said before, by default, usage data aggregation is enabled as you can see in the image below. You can choose which events you want to be captured. By default all 8 events are chosen, you should consider choosing only those which are required as more event logging will have impact on the performance.

Next you need to specify the log file location and maximum size of the log file to be maintained at a time. By default it is stored in a log folder of the "14 Hive" or the SharePoint root folder along with trace files. You should consider changing this location to some other drive than the drive on which SharePoint has been installed. (A high speed drive might be good, especially in case you are using verbose logging).

Also make sure that same location is available on all the servers of the farm.


When you scroll down the page below, you will notice that even health data aggregation is also enabled by default. You can change the default Log Collection Schedule or Health Logging Schedule by clicking on either of these links. Each element can be individually scheduled.

Next you need to specify the server name, database name and authentication details to connect to the database for logging. It is recommended to have the SharePoint logging database on a separate server or separate drive with its own spindles. By default database name is WSS_Logging.

When you click on the Health Logging Schedule link as shown in the image above, you will be taken to another page (shown below). Notice there are several timer jobs with different schedules to collect the different health related data. Again the more logging you do or the more frequently you do it, there will be an impact on the performance and you should change it as per your need. You will notice that it does not only capture data related to SharePoint but also capture SQL Server related events as well.

Now connect to the server and database to which you specified above.

By default the database name is WSS_Logging which has several partitioned tables as shown below:

Don't worry about writing complex queries from these partition tables. There are several views built on top of these partition tables to simplify your queries.

You can query these views as shown below. Please note these views have a column called CorrelationId about which I talked in my last tip. The Correlation Id is a GUID which gets assigned to each log entry. When there is any error on the page SharePoint 2010 shows this correlation Id on the page. You can take this correlation Id and do a search to find the reason for failure. Now the best part is the Correlation Id is grouped on the same conversation/session (initiated by a user) on the server and across the servers so that you can relate all the related log entries together. Also the correlation Id is not generated in case of an error only, but for all the conversations, which helps you to troubleshoot a slow loading of page as well.

Notes

  • The SharePoint logging database is fully supported for reading, querying or even writing data back to it unlike other databases of SharePoint.

  • The capturing of usage and health data collection is enabled by default.

  • It is recommended to have the SharePoint logging database database on a separate server or separate drive with its own spindles. By default database name is WSS_Logging.

  • By default, information is logged in a Log folder under the root folder where SharePoint has been installed. You should consider changing this location to some other drive than the drive on which SharePoint has been installed (and consider a high speed drive especially in case you are using verbose logging). Also make sure that the same location is available on all the servers of the farm.

  • A developer can write custom messages for diagnostic logging using the SPDiagnosticsService class. Not only that, you can even control the diagnostic logging behavior programmatically using this class. Please note it does not work in a Sandboxed environment (unless you create a full trusted proxy and call it from the Sandboxed solution). There is an unsupported ULS log viewer which you can use to view the log in a more user friendly interface instead of using standard log file viewers, click here for more details.

Business Data Connectivity metadata store is currently unavailable

Problem

The SharePoint 2010 Business Data Connectivity Service allows connecting with external data systems (WCF service, database, .Net assemblies). This is an improved version of the Business Data Catalog in SharePoint 2007. But you can get an error...

"Business Data Connectivity metadata store is currently unavailable"

...when navigating to the External Content Types using SharePoint Designer 2010.

Solution

You can get this type of error from Windows Server 2008 or the Client OS (Windows 7) after installing SharePoint 2010 and SharePoint Designer 2010. SP Designer can throw the following exception when navigating to an External Content Type object.


Step 1

This problem can happen if your SharePoint is installed in the Windows 7 OS. You can solve this by installing the Windows6.1-KB976462-v2-x64 Hot fix. Another reason for this error would be that the (BDC) service isn’t running properly as described in Step 2.

Install Windows6.1-KB976462-v2-x64

This should resolve the BDC error.


Step 2

For Windows Server 2008, ensure all updates are installed using Microsoft Windows Update. If the the problem persists again after restarting the PC then go to Manage services on the server in SharePoint Central Administration and check the BDC and the Metadata service.

  • Central Administration –> System Settings –> Manage services on server
  • Start the Business Data Connectivity Service and the Managed Metadata Web Service if they are stopped.
  • Close SharePoint Designer and re-open again.

The above error should be resolved after starting SharePoint Designer.

Load Balance in sharePoint 2010

am logged on to LAB-INDX as PINTOLAKE\Service-SharePoint. LAB-INDX is one of the servers where Central Administration is located, to avoid double hops. I will configure the farm directly on the server which hosts Central Administration

Open “SharePoint 2010 Central Administration” from the “Microsoft SharePoint 2010 Products” under All Programs

Select “Configure alternate access mappings”

On the right side of the screen pull down the menu next to “Alternate Access Mapping Collection”

Select “Change Alternate Access Mapping Collection”

Select “Central Administration”

If you notice both of the servers that we added the Central Administration web site to are present, however if you look that the URL on the right. Its doing a redirect to http://lab-indx:22222. Which is not what we want to do.

Click “Edit Public URLs”

The Default is “http://lab-indx:22222″. I added “http://lab-indx-mr:22222″ for the Intranet and “http://spca.pintolake.net” under Custom

CUSTOM is optional; it will require a DNS entry pointing “spca.pintolake.net” to the Load Balanced Virtual IP Address of the two indexing servers (LAB-INDX and LAB-INDX-MR)

Now our Public URL’s for the Zone are correct.

If you did not enter a “Custom” zone, you do not need to do the next step. At this point you will be able to access the Central Administration web site via “http://lab-indx:22222″ and “http://lab-indx-mr:22222″ which should be fine in most cases.

Edit HOST HEADER in IIS

When you create an Alternate Access Mapping it does not change the Bindings in IIS, we need to manually edit the bindings on all servers that are hosting the web application for that host header entry.

We will repeat the “Edit HOST HEADER” process on both servers which you are hosting the Central Administration site. In this case I am making this change on LAB-INDX and LAB-INDX-MR

Open “Internet Information Services (IIS) Manager” from “Administrative Tools”

Right Click the “SharePoint Central Administration v4″ site and select “Edit Bindings…”

Click “Add…”

Enter “spca.pintolake.net

Press “OK”

Click “Close” and exit IIS Manager

Repeat the “Edit HOST HEADER” process on both servers which you are hosting the Central Administration site. In this case I am making this change on LAB-INDX and LAB-INDX-MR

Testing

Try to access Central Administration site from all three URL’s. If you cannot access the Load Balanced IP it could that you have to enter the DisableLoopbackCheck on all of your SharePoint 2010 servers including the SQL Server.

I was able to access the Central Administration web site via:

Perform NLB testing on http://spca.pintolake.net by disabling NLB on each server then testing access from a remote location.

Server and Site Architecture SharePoint 2010

Depending on the type of custom application or solution that you are creating, you use different entry points into the object model to obtain the appropriate object from which to start. For example, if you are customizing administration and configuration of a deployment, you can use the static ContentService property to return the current Web service object and its collection of Web applications. To modify settings in the administrative Web application, instead use theAdministrationService property. Collection classes that derive from the Microsoft.SharePoint.Administration.SPPersistedObjectCollection(Of T) class inherit a GetValue method that you can use to return a specific object from a collection.

Note Note

If you are creating a Web Part, custom Web service, or Web application to work with site collections, individual sites, or lists, you can use members of the Microsoft.SharePoint.SPContext class to obtain the current site collection, Web site, or list. When you create a Web application in the /_layouts virtual directory, its functionality becomes available to all sites on the Web server. Outside of an HTTP context, such as in a console application or a Windows application, use a constructor of the SPSite class to obtain a specific site collection and to reach various objects within the collection. For more information, see Getting References to Sites, Web Applications, and Other Key Objects.

The following diagram shows the SharePoint Foundation server architecture in relation to the collections and objects of the Microsoft.SharePoint.Administration namespace.

Server architecture and Administration namespace
  1. The SPFarm object is the highest object within the SharePoint Foundation object model hierarchy. The Servers property gets a collection representing all the servers in the deployment, and the Services property gets a collection representing all the services.

  2. Each SPServer object represents a physical server computer. The ServiceInstances property provides access to the set of individual service instances that run on the individual computer.

  3. Each SPService object represents a logical service installed in the server farm. Derived types of the SPService class include, for example, objects for Windows services, such as the timer service, search, the database service, etc. and also objects for Web services, such as the basic content publishing Web service which supports the Web applications.

  4. An SPWebService object provides access to configuration settings for a specific logical service or application. The WebApplications property gets the collection of Web applications that run the service.

  5. If the service implements the Service Application Framework of SharePoint Foundation, then it can be split into multiple configured farm-scoped instantiations (CFSIs). Each of these provides the functionality of the service but each has its own individual permission and provisioning settings.

  6. Each instance of a service, or a CFSI, that is running on a specific server is represented by an SPServiceInstance object.

  7. An SPDatabaseServiceInstance object represents a single instance of a database service running on the database server computer. The SPDatabaseServiceInstance class derives from the SPServiceInstance class and thus inherits the Service property, which provides access to the service or application that the instance implements. The Databases property gets the collection of content databases used in the service.

  8. Each SPWebApplication object represents a Web application hosted in an Internet Information Services (IIS) Web site. The SPWebApplication object provides access to credentials and other farm-wide application settings. The Sites property gets the collection of site collections within the Web application, and the ContentDatabases property gets the collection of content databases used in the Web application.

  9. An SPContentDatabase object inherits from the SPDatabase class and represents a database that contains user data for a Web application. The Sites property gets the collection of site collections for which the content database stores data, and the WebApplication property gets the parent Web application.

  10. An SPSiteCollection object represents the collection of site collections within the Web application.

The following diagram shows the SharePoint Foundation site architecture in relation to the collections and objects of the Microsoft.SharePoint namespace.

Site Achitecture
  1. Each SPSiteobject, despite its singular name, represents a set of logically related SPWeb objects (see below). Such a set is commonly called a "site collection," but SPSite is not a standard Microsoft .NET collection class, in contrast to SPWebCollection. Rather, it has members that can be used to manage the site collection. The AllWebs property provides access to the SPWebCollection object that represents the collection of all Web sites within the site collection, including the top-level site. The SPSite.OpenWebmethod of the SPSite class returns a specific Web site.

  2. Each site collection includes any number of SPWeb objects, and each object has members that can be used to manage a site, including its template and theme, as well as to access files and folders on the site. The Webs property returns an SPWebCollection object that represents all the subsites of a specified site, and the Lists property returns an SPListCollection object that represents all the lists in the site.

  3. Each SPList object has members that are used to manage the list or access items in the list. The GetItems method can be used to perform queries that return specific items. The Fields property returns an SPFieldCollection object that represents all the fields, or columns, in the list, and the Items property returns an SPListItemCollection object that represents all the items, or rows, in the list.

  4. Each SPField object has members that contain settings for the field.

  5. Each SPListItem object represents a single row in the list.