Using the Content Editor Web Part to Query list from other sites through SharePoint Web Services

 

                So I recently ran into a situation where I had a client who had a hosted solution that did not allow for the Content Query Web part or any custom web part or component coding. All we had was the content editor web part and some creative JavaScript coding to help us.

                The goal was to display data from a list located on the root site on each of the 35 subsites in the farm without custom coding, without having to recreate and maintain the same list or make any linked lists, lookup fields, etc on all of the subsites.  I found a number of SPS03 sites that got me nearly where I needed to be. I took their code and converted it to a format that matched what MOSS returned from the web service call.

The solution was to insert the code below into the source editor of a content editor web part:

<span id=’uniquecontrolName'></ span>

<script language=javascript>

getListList();

 

function getListList() {

  var txt = document.getElementById(' uniquecontrolName ');

 

  //Build SharePoint Web Service URL based on current location

  var wsURL;

  wsURL = window.location.protocol+"//";

  wsURL += window.location.host;

  var path = window.location.pathname.split("/");

  path.pop();

  var x;

  for (x in path) {

    wsURL += path[x] + "/";

  }

  wsURL = "http://something.something.com/_vti_bin/lists.asmx";

 

//new soap action and xml

 var wsSoapAction = "http://schemas.microsoft.com/sharepoint/soap/GetListItems";

 var wsXML = '<?xml version="1.0" encoding="utf-8"?>';

 wsXML += '<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">';

 wsXML += '<soap:Body><GetListItems xmlns="http://schemas.microsoft.com/sharepoint/soap/">';

 wsXML += '<listName>List name or List GUID</listName>';

 wsXML += '<query></query>';

 wsXML += '<queryOptions><QueryOptions><IncludeMandatoryColumns>TRUE </IncludeMandatoryColumns><DateInUtc>TRUE</DateInUtc><viewFields><ViewFields><FieldRef Name="Event%20Date" /></ViewFields></viewFields></QueryOptions></queryOptions>';

 wsXML += '</GetListItems></soap:Body></soap:Envelope>';

 

//Create XML Document and get HTTP response using XMLHTTP object

var xmlDoc = new ActiveXObject("Microsoft.XMLDOM");

try

{

var httpResponse = getServiceResults(wsURL, wsSoapAction, wsXML);

if (parseInt(httpResponse) == 404) {

    txt.innerHTML = "<p>This code can only be executed from a web part.</p>";

    return;

  }

  else {

    xmlDoc.loadXML(httpResponse);

  }

 

}

  catch(e) {

    alert(e.message);

  }

 

//If getServiceResults returns a 404, then it's probably because the

//page is being launched from a document library instead of a web part

  if (parseInt(httpResponse) == 404) {

    txt.innerHTML = "<p>This code can only be executed from a web part.</p>";

    return;

  }

  else {

    xmlDoc.loadXML(httpResponse);

  }

 

  //Get results into collection

  listitems = xmlDoc.getElementsByTagName("z:row");

 

  //Loop through results and build table rows

  var output = "";

  for (var x = 0; x < listitems.length; x++) {

 

    output += "<tr>";

    output += "<td>" + listitems(x).getAttributeNode('ows_Event_x0020_Date0').text + "</td>";

    output += "<td><a href='http://privateplacement.edensandavant.com/Lists/Calendar/DispForm.aspx?ID=" + listitems(x).getAttributeNode('ows_ID').text + "'>" + listitems(x).getAttributeNode('ows_Title').text + "</a></td>";

    output += "</tr>";

  }

 

  //Display table

  var table = "";

  table = "<table border='0' width='100%' cellpadding='2' ";

  table += "cellspacing='0' class='ms-summarystandardbody' rules='rows'>" ;

  table += output;

  table += "</table>";

  txt.innerHTML = table;

}

 

function getServiceResults(url, soap, xml) {

  //Send XML packet to web service and return HTTP response text

  try {

    if (xml.length > 0) {

      xmlHttp = new ActiveXObject("Microsoft.XMLHTTP");

      xmlHttp.open("POST", url, false);

      xmlHttp.setRequestHeader("SOAPAction", soap);

      xmlHttp.setRequestHeader("Content-Type", "text/xml");

      xmlHttp.send(xml);

      if (parseInt(xmlHttp.status) == 404) {

        return 404;

      }

      else {

        return xmlHttp.responseText;

      }

    }

  }

  catch(e) {

    alert(e.message);

  }

}

 

</script>

       So after numerous installs, I ran into an issue with this install that totally hosed a number of servers. The boxes affected were all 32 bit MOSS 2007 enteprise machines, single Server dev implementations using SQL Server 2008 om Windows Server 2008.

 

      On step 8 it would consistently fail. The upgrade logs indicated a conflict in one of the search databases:

"ReflexiveUpgrade [SPServer Name=<machinename> Parent=SPFarm Name=<configDBName>] failed.
[SPManager] [ERROR] [6/17/2009 12:55:20 AM]: An update conflict has occurred, and you must re-try this action. The object SPSearchDatabase Name=<WSSSearchDB Name> Parent=SPDatabaseServiceInstance is being updated by domain\installaccountname, in the OWSTIMER process, on machine <machinename>.  View the tracing log for more information about the conflict.
[SPManager] [ERROR] [6/17/2009 12:55:20 AM]:    at Microsoft.SharePoint.Administration.SPConfigurationDatabase.StoreObject(SPPersistedObject obj, Boolean storeClassIfNecessary, Boolean ensure)"

 

    I tried the solutions posted here:

 

 and other places with no luck. Could not get the config wizard to finish. Out of frustration, I ran the wizard 1 more time, as soon as it hit step 8 I clicked the "cancel" button and allow the config wizard to stop in a more controlled manner. I reran the config wizard right after that and it ran to completion!

 

  Cannot say at this point in time I have any idea wh this worked, what was broken, but in addition to the solutions provided at the link above (which do work for many folks), this is another option to try.

 

MOSS 2007: Meeting DOD DISA STIG restrictions with your farm

 

                Throughout my experiences with MOSS, I have done more architectures, configurations, installations, than I can count. I have had many challenges in farm configuration but none of these come close to the challenge of meeting DOD DISA STIG restrictions (listing: http://iase.disa.mil/stigs/stig/index.html)  for a MOSS farm. There were many challenges, some technical some organic in nature. I am going to attempt here to give you a strategy for approaching this task, some lessons learned, and where possible some areas you need to be careful of.  Anyone who has gone through this and would like to share, PLEASE DO. This can be quite a challenge.

                First off, when it comes to STIGs. If you have looked at the list, they are not a “Change setting A at registry point A” type of listing. In many cases, there are different ways to achieve compliance with a particular item. A good example, had a client who needed to correct an issue with the VB Scripting engine on the OS. The way MS recommends doing this, is to restrict security (ACLs) on the dll. The client’s methodology was to unregister the dll. This caused serious issues for MOSS when it came to configuration and patching as this dll was accessed by the configuration manager.

                Adding to this, currently Microsoft does not provide any official guidance on implementing the STIGs set. So most companies to not have that guidance on how to implement these without disabling operating system components.

Preparation

                When you start down this road, do not assume that just because in one house you ran through compliance with no issue, that you will have no problem at another one. First recommendation here, from the onset, get a virtualized environment that matches the production environment as much as possible. It MUST at least have the same operating system version and patch level.  Get this designed up from and in place ASAP. Your life will be much easier if you sort the STIG issues out up front for the particular environment than on the end of it.  It is likely, if you are in the DISA STIG environment you will have some firewall restrictions as well. It is really best if you also configure your test environment to match the same firewall restrictions you will have in production.

                Second recommendation, patch your MOSS environment (OS and applications) up to the latest  service pack and cumulative update. There are some restrictions (such as disabling SSL v2.0) which are much easier to correct when your OS is fully patched.

                Before we move on to planning your process keep this is mind, MOSS needs more permissions during the installation, and configuration process than it does during normal operation. The configuration wizard will utilize ports and OS components that you may not use otherwise. For this reason you need to be sure your restrictions are tested against the MOSS configuration wizard, SSP configuration, Web Application configuration, etc as well as just a normally operating MOSS environment.

 

Process

                Third recommendation, plan out your process for reaching compliance before you start.  You will need to plan on getting time from the networking, platform (OS), AD, and application security staff during this process. This may be 1 person or a dozen people depending on the size of your organization.

                As for the process, one method which worked for us on a previous engagement. Apply the full set of STIGs for the company to your environment prior to installing MOSS. Once this is done, attempt to install and configure your application/Index server. Chances are this will fail in some manner. 

                Assuming it does fail, the next step, break the STIGS up first by severity level (High, Medium, Low, Informational). For each of these groups, break the STIG up further into groups of 20-30. Try to group them into functional areas (i.e. File ACLs, IE settings, service permissions, etc). If a particular group holds lower risk of causing issues you can group more than 20-30. A good example of this. We had 80 IE settings that posed an extremely low risk of causing MOSS issues, we grouped all together.

                For each grouping, first apply the full group, and test the config/install you tried with the full set. If no issues occur, snapshot your VM, and move on to the next group. If the group causes an issue, roll back and move through the group 1 by 1 till you locate the setting that causes the issue.  Depending on staff availability, the manner in which certain STIGs are applied, and farm architecture, it can take between 1 to 4 weeks to run through 700 settings in this manner. 

                Keep in mind, the vast majority of settings will not impact MOSS. In the last project we worked through out of 600+ settings 10 caused issues. You will need to refine your process to find issues like this on an ongoing basis as STIGs are periodically updated and you will find the need to continually test these new restrictions. Although, the updates will come in much smaller chunks than the initial implementation (maybe average 6-20 additional settings every quarter).

                One other note, reboot at least once after EACH group is applied. Some of these restrictions will not be implemented fully until you reboot. You absolutely need to reboot after each set of restrictions are thrown on the system.

 

Areas of concern

                While the implementation of STIGs are to an extent unique to the site. There are some areas of concern that have been encountered clients sites that I can provide a heads up on that may allow you to head off some issues:

1.       File ACL’s – This is one of the biggest areas of concern. Many of MOSS’ issues here are the same for any ASP.Net based applications. Of particular concern  are the folders used for the ASP.Net JIT compiler and .Net framework assemblies. Many of these are located in c:\windows subfolders and are accessed with user accounts that are NOT in the administrators group. It is HIGHLY recommended you do not just throw all MOSS service accounts into the administrators group. This is in violation of security best practices for MOSS and can lead to other problems. In addition to the c:\windows there is also another important folder at: C:\Program Files\Microsoft Office Servers\12.0. Under this folder, you have the office server web services, on query servers you also have file shares (by default) than contain your search indexes, as well as other resources your farm will need. See references for technet article on MOSS File permissions.

2.        Component Services/DTS – You will have a hard time finding anyone at MS (even the product team) to tell you any more than “I don’t know” about whether component services and DTS are necessary for MOSS to function. After much debugging and sleepless nights, I can say without it functional, your MOSS farm will not function properly. Whether it is component services or an underlying component,  when this goes down particularly Search begin to malfunction. This also seems to impact the Adobe Ifilters as well. Generally, registry or file ACLs are the culprits for bringing this service down.

3.       Local Security Policies

a.       Log in as a service – Some MOSS service accounts (depending on configuration) need to be able to log in as a service (i.e. web app pool accounts, search service accounts, excel services, etc). When these right are revoked, MOSS services will fail to start on reboot.

b.      FIPS – FIPS is a requirement by many government agencies. However, it is not a throw the setting and life is good with MOSS. You will need to make adjustments to your servers, web apps, and moss configuration to implement it. See the FIPS references to get started on this.

4.       Registry ACL’s – Registry ACLs can cause issue across the board to just about any component in the operating system or MOSS. You need to carefully inspect any registry ACL’s that are being applied. References for technet article on MOSS registry permissions

 

Summary

                Hopefully, this will give you a good start on implementing the DISA STIGs on your environment and help you avoid some pain. Please feel free to post any comments/links, etc from your own experience. This can be a daunting challenge and until we have good guidance from MS on securing the platform for these, any community assistance is greatly appreciated.

Reference

1.     MOSS File/Registry permissions: http://technet.microsoft.com/en-us/library/cc721638.aspx

2.     Firewall rules: http://blogs.msdn.com/joelo/archive/2007/02/13/protocols-ports-and-firewall-rules.aspx

3.     FIPS References: http://support.microsoft.com/kb/935434, http://support.microsoft.com/kb/811833

                In such an environment, implementing MOSS can be an extreme challenge to say the least. Using some hindsight looking back at the challenges the project encountered though, I can say a lot of the risks can be mitigated somewhat with some forward planning. I will first give an extremely high level description of the farm, and I will break down the risks into 2 main groups (in no particular order), either of which can severely impact your architecture. There is way too much to go in depth on everything, I will however provide links to help you out if you need to face such a project.

Farm Layout

                The farm we created had 4 WFEs, I App/Index/CA server and a clustered SQL 05 (A/P) on the back end. It had only 5 Site collections, 1 of which had to be broken off to its own Web app using a separate SQL instance on the cluster for its content DB (for security/compliance reasons).  It utilized standard NTLM AD auth for the farm with OOB search.  The WFEs were separated out into 2 groups of 2 with each group of 2 having their own load balancer and host header. The entire system was based on 64 bit architecture.

Networking/Connectivity

                In our implementation, we had some pretty heavy firewalls around each of the servers. This included firewalls around the network, one for each MOSS server and one for the SQL cluster. Needless to say we had ,to be pretty exhaustive on a list of firewall rules. All told, we have 70+ firewall rules to allow the farm to talk.

                To this point, you need to consider a few things. First of all, find a tool to independently test each rule to ensure it is open, and to trace what communication MOSS is attempting. I have seen a lot of references on MOSS ports online, NONE of them hit all the ports we needed. No matter how good you are, it is likely you will miss at least a couple. 

Also, keep in mind that MOSS uses different ports depending on what you are doing.  For example a MOSS farm already configured, running normally will use a smaller set of ports than one that is being actively configured or patched. For example, we found all calls to the DB (UDP 1433, 1434) that only occurred during the config wizard and never any other time.  Also MOSS will utilize different ports depending on if it uses Kerberos or SSL, or LDAP/AD, or depending on how your SQL cluster is configured.

You would be wise to have a nice, easy to read chart on all the servers and services you need to hit, particularly your non-MOSS servers such as SQL, AD servers, DNS Servers, SMTP, etc. They will likely be firewalled as well.

 IMO, you need to resist the temptation to just grab a huge list from the web and push all the rules it gives you out there. It is a fast track to getting your farm up but in the long run, such a game plan will leave you opening up holes in your firewall that you will never use. As painful as the firewall can be, it is there for a reason and you do not want to open up security holes that can leave you or your client failing an audit.

You will want to sit down with the networking people and explain everything you are looking for up front. Be extremely anal. Down to the specific host headers (in quotes), ports protocols and be prepared to explain in deep detail why each port is used. Any network engineer in such an environment that is worth his salt will ask for this info. Also, set the expectation with them that while you will try to nail as many rules as possible, you will miss some and you will need to define the process that will allow you to add additional ports. Most network admins will appreciate that you want to open as few as possible at first and then slowly open additional ones as you need them.

Links:

http://blogs.msdn.com/joelo/archive/2007/02/13/protocols-ports-and-firewall-rules.aspx

 

File permissions

                In our implementation, our server OS’ also had some serious compliance requirements for the DOD restrictions. This meant the compliance software which pushed these rules removed tons of ACLs from the file system and registry, disabled services, blocked ports, reset IE permissions, and a few other fun things. You would be wise to ask for a list of these up front and exclusive obtain exclusive access to the person responsible for maintaining the list of restrictions. Depending on the scope of your restrictions, finding the resolution to this can involve accessing multiple MS product teams or reverse engineering ACL, service, and Group Policy permissions required for MOSS, WSS, IIS, Asp.Net, .Net (all versions), COM+, and even Windows Server 2003 itself. There were rules in there that actually killed the OS.

                Keep in mind, if you have DOD compliance, you will need to provide detailed justification for each and every exclusion you identify. Usually, it has to be more in depth than “It breaks ASP.Net”. There a few good links I found on technet to at least get you moving on this. However, if you are doing this for the first time for any of the dependencies for MOSS (IIS, Asp.Net, etc), you would be wise to call MS premier support and proactively get them involved in helping you debug this. Also, you will need someone above the front line offshore support, they will not be able to help you in any manner on this. It is simply over their skill set.

                Aside from MS support, the only other way I have found, is to get a functional system up and compare the permissions as they cause issues. It is a long, painfull process so buy some coffee and stock in whatever company makes it. You will be needing it if you take this route. Also, if at all possible, virtualize the machine you are testing with. Rebuilding a physical machine over and over will cost you weeks.  As with the firewalls, you need to resist the urge to just grant cascading permissions all over the place. You need to just open up what you need and nothing more. Particularly in the c:\windows folder. The DOD restricts these for a reason.

Links:

http://technet.microsoft.com/en-us/library/cc721637.aspx

http://technet.microsoft.com/en-us/library/cc560966.aspx

http://technet.microsoft.com/en-us/library/cc678863.aspx

http://technet.microsoft.com/en-us/library/cc721638.aspx

Listing Site Collections with any User - RunWithElevatedPermissions is your friend

 

So I had a web part I wrote a long time ago that would list sites and subsites on a given SPWeb as deep as you wanted to go and allow a good deal of flexibility on how they are displayed. Recently, I had the opportunity to modify it to scan a list of site collections in a given Web application.

We had a setup with 80+ site collections for a clients extranet. Each site collection is a separate project for their clients. Some clients had multiple site collections and each of these had the need to explicit security encapsulation. So some users did in fact have access to multiple site collections, most would have 1 or maybe 2.

The issue with this, the standard scanning code I had could not jump site collections. The only code samples I managed to find, involved using the Microsoft.SharePoint.Administration namespace. The issue there being, it would work fine for a MOSS administrator, however when another user hit the page with the web part, they would be given a generic MOSS Access Denied page.

The answer I got came from merging a few posts I found while frantically googling some way to do this. It involves first using the RunWithElevatedPermissions directive to run under an account that has privileges to access all the site collections. The second part involves checking the DoesUserHavePermissions function off the root web on the site collections.

It is not a huge breakthrough but if it helps save someone some time it is worth it. See code below:

 

  SPSecurity.RunWithElevatedPrivileges(delegate()

  {

      SPWebApplication webApplication = SPWebApplication.Lookup(serverUri);

      SPWeb rootweb;

 

      foreach (SPSite siteCollection in webApplication.Sites)

      {

         rootweb = siteCollection.OpenWeb();

         //do not add it is the current user does not have access to the site

         if (rootweb.DoesUserHavePermissions(objOriginalUser.Name,    

            SPBasePermissions.ViewPages) == true)

            {

             if (rootweb.Url.Contains(startingPointURL + "personal/") == false)

             {

                 userWebInfo = new clsUserWebInfo();

                 userWebInfo.ID = rootweb.ID;

                 if (rootweb.Name.Trim() == "")

                 {

                     userWebInfo.Name = rootweb.Title;

                  }

                  else

                  {

                     userWebInfo.Name = rootweb.Name;

                  }

                                                        userWebInfo.Description = rootweb.Description;

userWebInfo.Title = rootweb.Title;

userWebInfo.URL = rootweb.Url;

alUserWebs.Add(userWebInfo);

}

               }

             }

          });