MOSS 2007, Capacity Planning, Performance Summary, Optimal Topologies and HP Servers

HP has recently published a white paper titled “Microsoft Office SharePoint Server 2007 on HP ProLiant Servers – Performance Summary” that has been authored by Jimi Ibbett. The paper is available here:

http://h20195.www2.hp.com/V2/GetDocument.aspx?docname=4AA1-1793ENW&cc=us&lc=en

For any enterprise server like MOSS, capacity planning for the required performance levels is always tricky. It’s difficult to get optimal capacity without testing different types of loads on various configurations of different hardware. This paper has been published after testing various workloads on 12 different servers which were mix of HP Proliant Blade and Rack servers. The servers had various combination of  Processors – single core and dual core, AMD and Intel, x86 and x64.

You can expect to have answers for the following questions (but not limited to these) after reading this paper:

• What are best practices or example configurations for different types of environments: entry level, highly available and application optimized?
• What kind of performance can be expected for example configurations for different types of workloads (read only/read write) for parameters such as Throughput, CPU utilization for Web Frond Ends(WFE) and Database (DB) servers?
• How 32 bit and 64 bit processors are different in terms of performance and what’s HP recommendation?
• What are general rules of thumb one can follow for MOSS performance planning? One example thumb rule provide in the paper is:
  “For the 2-server SMB configuration (all Office SharePoint Server services running on a single server), assume a maximum throughput of 75-100 requests/second. For a single-server SMB configuration (All Office SharePoint Server services and SQL Server running on one server), assume a maximum throughput of 50-75 requests/second.”

• What should one do for IIS application pool tuning and memory usage?
  How different types of Caches provided in MOSS be used to get optimal performance
• What kind of network traffic be expected between different servers?
• How do you size your solution for expected throughput?
• How to determine required Throughput?
• How to estimate storage for different MOSS servers?

You’d also get to know some interesting facts such as:
The performance (server throughput) of Office SharePoint Server 2007 will be less than SharePoint Portal Server 2003 when characterized on the same type/class of server.

SharePoint 2007 (MOSS) Enterprise Search and BDC – QuickStart Resources

ssRecently, I had to get a few of my team members ready quickly for implementing Search in MOSS to search in multiple content sources (Oracle, Lotus Notes, SQL Server). So I prepared a list of URLs about the information available publicly and that would be sufficient to get started with. Sharing those URLs here:

General Search

• Overview Article: Find It All with SharePoint Enterprise Search

http://www.microsoft.com/technet/technetmag/issues/2007/01/Search/default.aspx  

• TechNet Webcast: An In-Depth Look at SharePoint Server 2007 Search Technology (Level 200)  Event ID: 1032309959

http://msevents.microsoft.com/cui/WebCastEventDetails.aspx?culture=en-US&EventID=1032309959&CountryCode=US

• TechNet Webcast: Installing and Configuring Search in SharePoint Server 2007 (Level 300)  Event ID: 1032325467

http://msevents.microsoft.com/cui/WebCastEventDetails.aspx?culture=en-US&EventID=1032325467&CountryCode=US

• Enterprise Search in MOSS Overview @ MSDN

http://msdn2.microsoft.com/en-us/library/ms495418.aspx

• Operations/How-To: Add a content source for search (Office SharePoint Server)

http://technet2.microsoft.com/Office/en-us/library/82c7c354-6347-4ae8-b5f8-7d0cdfe432401033.mspx?mfr=true

BDC and Search

• BDC Overview Video

http://channel9.msdn.com/ShowPost.aspx?PostID=221016 

• What is the Business Data Catalogue?

http://blogs.msdn.com/martinkearn/archive/2006/10/05/What-is-the-Business-Data-Catalouge_3F00_.aspx

• BDC Overview and Links at MSDN

http://msdn2.microsoft.com/en-us/library/ms546541.aspx

• TechNet Webcast: Extending SharePoint Server 2007 Search Capabilities by Exposing Data with the Business Data Catalog (Level 300)  Event ID: 1032325420

http://msevents.microsoft.com/cui/WebCastEventDetails.aspx?culture=en-US&EventID=1032325420&CountryCode=US

• Blog: BDC – Enabling Search on business data 

http://blah.winsmarts.com/2007-4-SharePoint_2007__BDC_-_Enabling_Search_on_business_data.aspx

• Tool: BDC Meta Man is the first application that provides the ability to create Application Definitions for the Microsoft Office SharePoint Server 2007 (MOSS) Business Data Catalog (BDC) without writing a line of code.

http://www.bdcmetaman.com/default.aspx

• Database Sample: AdventureWorksDW SQL Server 2005 Sample

http://msdn2.microsoft.com/en-us/library/ms494876.aspx

•  Web Service Sample: Walkthrough: Amazon Web Service Sample 

http://msdn2.microsoft.com/en-us/library/ms498501.aspx

• Enabling Business Data Search

http://msdn2.microsoft.com/en-us/library/ms492695.aspx

• BDC and Oracle Blogs:

http://www.sharepointblogs.com/sethnidhi/archive/2007/02/04/19004.aspx

http://www.sharepointblogs.com/ldusolier/archive/2006/12/22/17172.aspx

Search In Lotus Notes

•  Configure Office SharePoint Server Search to crawl Lotus Notes (Office SharePoint Server 2007)

 http://technet2.microsoft.com/Office/en-us/library/82c7c354-6347-4ae8-b5f8-7d0cdfe432401033.mspx?mfr=true 

• MOSS2007 – Search and Index Lotus Notes

http://www.sharepointblogs.com/helloitsliam/archive/2007/01/09/17654.aspx

• Using SharePoint 2007 to index a Lotus Notes Database

http://meiyinglim.blogspot.com/2007/01/using-sharepoint-2007-to-index-lotus.html

• Displaying correct Titles of  Lotus Notes Documents in SharePoint Search Results

http://meiyinglim.blogspot.com/2007/01/displaying-correct-titles-of-lotus.html

• How to Configure Search to Honor Lotus Notes Security Settings

http://support.microsoft.com/default.aspx?scid=kb%3bEN-US%3bQ288816#appliesto

Web Services, WS-* Specifications and Interoperability

For someone who has not been involved till now in any solution development which requires interoperability across web services developed on different platforms, it’s quite easy to believe that he/she can use web services specifications to achieve the required interoperability.  Because web services specifications and standards have been developed to get interoperability – so when one uses them, he should achieve it.

Unfortunately that’s not true. As far as the basic web services standards such as XML, WSDL, SOAP etc. are concerned, software vendors have been able to make their products interoperable to a great extent because of wide adoption of the WS-I’s Basic Profile. However, when one feels the need of advanced functionalities such as security, transactions, reliable message delivery in web services – many vendors are providing this through WS-* specifications such as WS-Security, WS-ReliableMessaging (WS-RM), WS-AtomicTransactions (WS-AT) etc. But when one needs to interoperate between web services being developed on different platforms using WS-* specifications – life is not that easy.

One might need to take care of many aspects. Even then, achieving interoperability might not be possible in some contexts. So whenever someone is designing a new solution that involves web services across different platforms – he need to provide special attention to selection of platforms and the way the solution is built.

I have recently worked on a project that involved three different platforms: Microsoft Windows Communication Foundation (WCF), BEA Weblogic Server and IBM Websphere Application Server and uses many WS-* specifications: WS-Security, WS-Federation, WS-RM, WS-AT. The experience was not very good as there was lot of tweaking and changes that we had to make to get different web services interact with each other; however we also learned quite a few lessons. I’ve documented these lessons in my recent article that is published in SOA Web Services Journal. You can read the article at the following location:

http://webservices.sys-con.com/read/314083.htm

Lifecycle Methodology Management Solutions and VSTS

Process Templates and lifecycle methodologies are one of my personal favorite areas in VSTS. So I’ve written another article on the topic in Sys Con’s The Dot Net Journal: Understanding the Methodology Workbench of Microsoft Visual Studio Team System – Underlying Concepts and Architectures

The article provides a starting point for those who are evaluating VSTS for managing and automating lifecycle methodologies in their organizations. While it provides information on what types of solutions are available for managing lifecycle methodologies and where VSTS fits in those solutions, it also explains how non Visual Studio based developers (for example java developers) can leverage the benefits provided by VSTS platform. Other topics that I covered are:

o       List of benefits and limitations of only methodology management area of VSTS

o       Introduction to Process Templates and its components

o       Implementing lifecycle methodologies using VSTS Process Templates

Happy
Reading!

Software Development Methodologies, VSTS and Process Automation

Software Development Methodologies are there for a long time. Though there are quite a few tools available for business process automation, there aren’t many options available for software processes automation. VSTS address this issue to a large extent. My article – “Software-Development Methodologies and Visual Studio Team System – From Documents to Automation” that is published on Microsoft Architecture site recently – describes how VSTS addresses process automation and what you need to do to automate your processes. Specifically it discusses:

• Need for automation
• Automation using VSTS
• Different phases of developing a new VSTS process template
• Analyzing typical life cycle methodologies and implementing their artifacts using Process Template elements

TFS, High Availability, Capacity Planning and HP

As an enterprise product goes RTM, its deployment options gain more focus. The customers who adopt that product need to pay attention to non-functional areas in addition to functional features of that product. Two key non functional areas are:

• High Availability
• Capacity Planning

This is true for Team Foundation Server also. We (HP) have been involved in all these areas with Microsoft right since early Beta stages of VSTS. When VSTS was in development stages, we extended TFS to develop an enterprise project and process management solution, APPRISE. Since TFS went RTM, we’ve been working on other two areas: Capacity Planning and High Availability.

For capacity planning, HP had released deployment configuration paper with HP Servers that i had written about in one of my earlier blog. This paper had given deployment configurations of different capacities in terms of numbers of users: 10-50, 10-500, 100-500, 500-3500.

For high availability, TFS Data Tier (DT) supported HA using SQL Server Clustering, however, AT HA configuration (using NLB) required manual intervention. Read Brian’s blog for more details on that. HP provided a sophisticated and automated solution for AT HA using HP Insight Manager. HP has released a white paper on this configuration also. This is a 53 pages paper that provides each and every step in detail:

• Configuring NLBs
• Installing and configuring TFS AT
• Installing and configuring HP SIM
• Configuring the RDP Job
• Configuring HP SIM and creating Tasks

Rob Caron has also blogged on these papers here. You can find HP’s VSTS related paper here.

And while testing TFS on HA setup was going on, we also tested our solution, APPRISE on the same setup and made it compliant to both the setups: SQL Clustering and AT warm stand-by failover. We demonstrated both TFS and APPRISE HA configurations in Tech Ed 2006.

VSTS and TFS Licensing FAQ

There is lot of confusion around Team Foundation Server (TFS) and Visual Studio 2005 Team System (VSTS) licensing. Although, Microsoft has published a detailed whitepaper (http://go.microsoft.com/fwlink/?LinkId=55933) around licensing, still I find many questions in VSTS Forums. So I’m writing the following FAQ that answers many commonly asked questions: 

Q1.  What are the different options available for using TFS?

At present, you can have one of the following options:

Type of TFS	Description	When shall I use?
TFS Trial Edition	This is available for free and works only for 180 from the day of installation.	You are evaluating TFS. Based on this evaluation you’ll decide to buy TFS Licensed version. For TFS Clients, you can use Visual Studio 2005 Team Suite Trial Edition, which is also available valid for 180 days.
TFS Workgroup Edition	It has identical functionality to TFS full version with the exception that it is limited to five users only. It is provided with Team System client products that are licensed with MSDN Premium Subscriptions.	You are a small team and maximum 5 users would be using TFS. Or you already have MSDN subscription that provides you Team Suite or Team Edition client – you want to pilot or test TFS in one team before investing in full fledged version. Also, you do not need Client Access Licenses (CALs) for those five users.
TFS	It is a full feature edition and needs to be bought separately. It does not have any limit on number of user or number of days.	You’ve already evaluated TFS and want to utilize capabilities with more and more users. You’ll need as many CALs as the numbers of users (or devices) accessing TFS.

Q2. How many TFS Server licenses I need when I’ve installed TFS in a dual server deployment mode (AT and DT on different servers)?

You need one server license for every server where TFS is installed. So you’ll need TWO TFS server licenses for a dual server deployment. However, you do not need extra CALs.  

Q3. If I’m using Data Tier (DT) in a SQL Server Cluster, do I need TFS Server license for every instance of cluster? Yes. For example, if you have deployed TFS in dual server mode, where AT is installed on one server and DT in installed on a SQL Server cluster with two nodes, you’ll need in total three licenses for TFS. In this case also, you do not need extra CALs Q4. Do I need additional TFS server license if I’m using TFS Proxy also?

Yes, you need full TFS license even for TFS Proxy. However, you do not need extra CALs. 

Q5. Do I need additional TFS server license for using TF Build? No, you do not TFS license for Build even if TF Build is installed on a different machine from a machine that hosts TFS. Also, you do not need Client Access License (CAL) for a device where TF Build is installed for server to server communication between TF Build and TFS. However, you need a CAL for users who are managing builds. Q6.  What are different types of clients available from Microsoft? At present, the following licensed Team Clients are available:

1. Visual Studio 2005 Team Edition for Software Architects
2. Visual Studio 2005 Team Edition for Software Developers
3. Visual Studio 2005 Team Edition for Software Testers
4. Visual Studio 2005 Team Suite 

Each of the above clients contains one CAL for TFS. So if a user has license for one of the client, he/she does not need additional CAL to access TFS. However, if you access TFS or SQL Server (where TFS is installed) in other ways such as with Team Explorer or through MS Project or MS Excel or with a third party – you need to additional CAL for each user. Only in the following situations, you don’t need addition CAL:

• View static TFS data that has been manually distributed outside of the server software e.g. MS Project or MS Excel files containing work items (Note that if you use these files to update or publish work items, you need CAL)
• Receive TFS reports through e-mail and do not indirectly access the server to refresh, create, or update data.
• Receive printed TFS reports.

Q7. There are users who access Team Project Portal in Windows SharePoint Services (WSS). Do I need CAL for such users?

As far as I understand, if they using WSS site in the following ways, you do not need
CAL:

• Accessing reports that have been manually distributed
• Accessing files containing static TFS data (such as MS Project or MS Excel files containing work items) for READONLY purposes. However, if your files are connected with TFS and  get refreshed you will need CAL
• Using WSS for any other WSS functionality (such as Threads, Events, Announcements, Lists etc.) 

Q8.  Do I need CAL if I just access Process Guidance through WSS Team Project Portal?

I think, Process Guidance is “static TFS data” – from that angle, a CAL is not required. But I’m little doubtful here. I would appreciate, if someone from Microsoft can clarify here. 

Q9. I heard that you need extra license for TFS Load Agent also. Isn’t the included in VS Team Edition for Software Testers?

Team Edition for Software Testers does not include the ability to “harness” multiple physical devices to generate additional load; load generation scenarios are limited by the hardware. Typically, for the recommended software configurations, this is approximately 1,000 virtual users (VU). If you need to simulate more VUs using multiple machines, you need to buy one Load Agent license for every 1000 VUs (i.e. capability of TFS Load agent on one machine).    Q10. Where can I Download Trial Editions?

• TFS Trial Edition: http://go.microsoft.com/fwlink/?LinkId=64135
• VSTS Team Suite Trial Edition: http://www.microsoft.com/downloads/details.aspx?FamilyId=5677DDC4-5035-401F-95C3-CC6F46F6D8F7&displaylang=en  

Q11.  If I use TFS Trial Edition or Workgroup Edition, would I be able to upgrade these editions to a full Edition

Yes, Microsoft support different types of upgrades for TFS. You can look here (http://msdn2.microsoft.com/en-us/library/ms404844.aspx ) for more details. 

Q12.  Where can I find more details?

The TFS licensing whitepaper (http://go.microsoft.com/fwlink/?LinkId=55933) published by Microsoft explains the licensing considerations and scenarios in detail. You should read that document before making your final decision about the TFS licenses. The pricing info is available at http://msdn.microsoft.com/vstudio/howtobuy/default.aspx.        

DISCLAIMER: The information provided here is just my understanding gained from licensing whitepaper (http://go.microsoft.com/fwlink/?LinkId=55933) and other publicly available information. I’m providing this with the intention of helping community. However, this should not be regarded as official FAQ from Microsoft. If you find any anomalies here, please point me, I’d correct those.  

What’s there in VSTS for Solution Integrator

Jeff Beehler recently posted a blog about the topics he plans to write in near future. I would like to see few blogs on opportunities of Solution Integrators. There are so much of documentation available on how to deploy and use VSTS. But what’s Microsoft’s plan. Do they expect customers to deploy VSTS on their own or they expect customers to engage Solution Integrators (SIs) for the purpose? However, is only deployment of development environment (can we address VSTS as that) large enough opportunity for SIs to pursue. Would they put money to define service offerings around this small opportunity? In this case, what a SI can offer?

– Physical Deployment of TFS (but how hard is it – would customers really buy that?)
– Customization of Process Templates as per customer needs
– Training workforce on using VSTS

Somehow, I feel that such a huge product with great extensibility options is being targeted to a smaller domain as compared to its capabilities. I feel Microsoft can do much better in selling VSTS, if it can come up with scenarios or solution that use VSTS as one key component of overall solution than just as a development environment. They can come up with many more examples like The Requirements Authoring Starter Kit that i wrote about in one of my earlier blog.

How to use Microsoft Project to manage projects with VSTS

I keep getting questions and comments for my article, “Combine Microsoft Project and Visual Studio Team System for a One-two Punch of Productivity”. The article is available at:

http://www.devx.com/dotnet/Article/30187

 

So, I thought of creating this blog entry, so that I can answer those questions here. The article covers the following topics:

 

• Problems in day to day Project Management activities
• How VSTS centralized architecture addresses those problems
• How do you create a new Project Plan and connect to VSTS
• How do you work with project tasks (creating, updating, deleting, resolving conflicts etc.)
• Column Mappings and Areas and Iterations
• Comparison with Microsoft Project Server

 

Happy
Reading!

VSTS and TFS from Solution Integrator’s perspective

As I’m from a Solution Integration and Services background, I always used to think what kind of solutions and services we can offer around VSTS. One option is to provide consulting, deployment and training services to the organizations that are planning to adopt VSTS. The other could be to provide a solution that is purely and completely based on VSTS and TFS such as APPRISE, an enterprise project and process management solution. But aren’t that too less options for a product that is extensible and customizable to such a great extent? 

TFS is such a feature rich product that it should be used in many more solutions than just in development related environments. Requirements Authoring Starter Kit is a perfect example of such a solution. The solution integrates and uses a number of Microsoft products including TFS:

• Microsoft Office Word 2003
• Microsoft SQL Server 2005
• Microsoft Windows SharePoint Services
• Microsoft Visual Studio 2005 Tools for Office
• Microsoft Visual Studio 2005 Team Suite
• Microsoft Visual Studio 2005 Team Foundation Server

The article about RASK introduced it as:

“The Requirements Authoring Starter Kit (RASK) provides a customizable requirements-authoring solution for software development teams. RASK serves two purposes. It provides the basis of a Requirements Authoring solution and illustrates how to access Microsoft Visual Studio 2005 Team Foundation Server programmatically from Microsoft Visual Studio 2005 Tools for the Microsoft Office System (Visual Studio 2005 Tools for Office). RASK has broad functionality that you can extend with minimal effort.” 

You may not use the solution for the functionalities it provides, but you may still like it to learn so many ideas like:

• How TFS can be integrated in a solution that comprises of so many products that are not related to development environments
• Get few more example on TFS extensibilty
• Another example of the capabilities of Visual Studio Tools for Office and Office
• …