SharePoint 2013 People Picker error: “Sorry, we’re having trouble reaching the server.”

I have had a strange error today in my dev lab environment. First I recognized that I could not select users from the People Picker:

image

Sorry, we’re having trouble reaching the server.

Second I realized that I was not able to use any function that belongs to WCF web services such as the SharePoint REST API, e.g. at http(s)://<server>/_api/web.

Using Fiddler I found this behavior:

image

HTTP 404 NOT FOUND on /_vti_bin/client.svc/ProcessQuery

Same for /_vti_bin/client.svc/web which is the same as /_api/web.

image

Search the web I found some hints regarding this error in SharePoint. But nothing worked.

Than I created a own web service “service1.svc” with a simple method in it and placed it in folder <SP-Hive>\isapi where the virtual folder “_vti_bin” is located on the file system.

Result:  I could not call my own web service too. Same result: NOT FOUND.

Than I search the web for “WCF 404” and found some hints to “HTTP Activation” feature of Windows Server OS. Of course this was activated for .NET 3.5 and .NET 4.5.

So I deactivated the “HTTP Activation” feature of .NET 3.5 and .NET 4.5 and re-enabled one by one (1st .NET 3.5, 2nd .NET 4.5 ) them after deactivation.

image

image

image

 

After that I did a IISRESET. – Than everything works again as expected. 🙂

image

and

image

Create User Profile Sync Connection for SharePoint 2013 by Script (A Supported Approach)

In my last post I wrote about a supported approach to create a user profile sync connection in SharePoint 2010 using PowerShell and Internet Explorer automation.

In the meanwhile I have adopted the script for SharePoint 2013.

Same conditions as last time: This is a kind of “Proof of Concept”. You have to modify it in order to use it in you scenarios.

 

Here it is:

http://gallery.technet.microsoft.com/Create-User-Profile-Sync-e9ec0cf7

 

Original article with some more details:

https://blog.kenaro.com/2013/09/11/create-user-profile-sync-connection-for-sharepoint-2010-by-script-a-supported-approach/

Create User Profile Sync Connection for SharePoint 2010 by Script (A Supported Approach)

Before we start…

1. It’s not supported to create a User Profile Sync connection by script or code using the server object model. We will NOT do this here!

2. The script in this article is part of my “dev lab” setup scripts. – The script cannot be used “as-is” in real world scenarios. If you want to use this approach in your deployment you MUST customize the script to fit your needs! – Especially for selecting sync entries from the Active Directory treeview selector.

3. The script is a Proof of Concept.

4. It’s tested only on SharePoint Server 2010. I’ll do it for SP 2013 later.

5. The script has almost no error handling !!! Sometimes I get the error message “MOSS MA not found”. In this case I restart the “FIMService” and try again.

 

The approach…

… is to use Internet Explorer as COM object to navigate to pages of the Central Administration and “act like a user”. This means:

1. Open IE window

2. Navigate to Central Administration -> Manage Service Applications

3. Search of the (first) “User Profile Service Application” in the Service Apps list and navigate to it’s settings page.

4. Open page “Configure Synchronization Connections”.

5. “Click” on link “Create New Connection”

6. Fill in connection parameters such as name, forrest, user profile sync account. Click button “Populate Containers”. Click “Select All”. Click “OK”. Wait until connection is created.

7. On the User Profile Service Application property page: Click on “Start Profile Synchronization”. There select “Start Full Synchronization”. Wait until sync starts. Wait until sync is complete.

 

The script…

 

Here it is:

http://gallery.technet.microsoft.com/Create-User-Profile-Sync-98ab7201

(Update) Here is the script for for SharePoint 2013: http://gallery.technet.microsoft.com/Create-User-Profile-Sync-e9ec0cf7 (Article here).

 

First the script contains some config settings for the new connection.

clip_image001

 

Appendix

Here are the input boxes related to the config values (line numbers):

clip_image003

The script is written in for a English localized CA. However I have created resource variables for other languages.

clip_image004

Usage of resource variables (line numbers)

clip_image006

clip_image008

clip_image010

clip_image012

clip_image014

[DE] MCM/MCSM/MCA abgekündigt – Ende einer Ära

Mein erster Blog Post auf deutsch…

Wie Ihr sicher bereits gehört habt wurde der Microsoft Certified (Solutions) Master und Microsoft Certified Architect von Microsoft am Freitag überraschend abgekündigt. – Nicht nur SharePoint: alle 4 Zertifizierungsprogramme wurden abrupt beendet.

Das ist die schlechteste Nachricht, die ich in meiner beruflichen Laufbahn je bekommen habe.

 

Voller Text hier:

http://blogs.technet.com/b/neiljohn/archive/2013/08/31/retiring-the-microsoft-master-certifications-and-training.aspx

 

Erste Artikel hier:

[von MCM Wictor Wilén] http://www.wictorwilen.se/microsoft-advanced-certification-mca-mcsm-mcm—the-end-of-an-era

[von MCM Paul Stork] http://www.dontpapanic.com/blog/?p=328

[von MVP Jeremy Thake] http://www.jeremythake.com/2013/08/mcm-certs-gone-microsofts-cloud-reality-distortion-field-in-full-force

http://www.theregister.co.uk/2013/08/31/microsoft_cans_three_pinnacle_certifications_sparking_user_fury/

http://www.zdnet.com/microsoft-abruptly-pulls-masters-certification-hints-a-replacement-may-come-7000020093/

 

Hier könnt Ihr Microsoft Eure Meinung sagen:

https://connect.microsoft.com/sqlserver/feedback/details/799431/please-dont-get-rid-of-the-mcm-and-mca-programs

 

Ich war noch im Juni diesen Jahres in Redmond zur Rezertifizierung für MCSM: SharePoint.  Das werde ich bis zum 1. Oktober nicht mehr abschließen können. – Und: Warum sollte ich es noch abschließen, auch wenn das möglich wäre? Sinn macht es so oder so nicht mehr. – Aber ernsthaft: 30 Tage Zeit für eine solche Aktion?! Meine (wenigen) grauen Haare verdanke ich den letzten 17 Monaten, in denen ich erst den MCM abgelegt und mich dann für den MCSM vorbereitet habe. – Besonders verstörend empfinde ich dabei, dass meine Kollegen und ich aus “R15” (“Rotation 15” = Zertifierungsdurchgang ab Juni 2013), letzte Woche (26.08. bis 29.08.)  noch in die Prüfung geschickt wurden obwohl den Verantwortlichen (offensichtlich) längst klar gewesen sein muss, dass die Zertifizierung Geschichte ist.

 

Eine Analyse, welche Hintergründe die Entscheidung haben mag, möchte ich an dieser Stelle nicht führen, denn das Ergebnis ist offensichtlich. Ich frage mich dieser Tage, wie weit Microsoft die “Cloud”-Strategie noch treiben wird. Ist es tatsächlich deren Ziel, beste Software-Produkte in den Wind zu schlagen und die Kunden quasi zu zwingen, sich Alternativen zu suchen? “Cloud” an sich ist nichts schlechtes, aber “nur Cloud” ist einfach Quatsch.

 

 

Set profile synchronization account rights in AD using PowerShell

This is just a short note on this topic. I could not find a script in the net. I’m not sure this is part of some SharePoint “config frameworks” like “AutoSPInstaller”. I do not use this frameworks. I create my own script sets. – Currently I create config scripts for my dev environment… The missing pieces here are related to User Profile Sync. Creating a sync connection *not* using Central Administration is not supported. There is no PowerShell, etc. – Another thing is to grant the sync account rights in the Active Directory… Here is my small script in Technet Gallery:

http://gallery.technet.microsoft.com/Set-SharePoint-profile-1a3d1283

The script will not modify an existing entry for the profile sync user in the AD. If the user is in the ACL of the Domain object or the Configuration object it will skip this part!

As always: Use it at your own risk!!! 

image

Use it like this:

image

 

Tested with SharePoint 2013 and a Active Directory based on Windows Server 2012.

Client Side Encryption of List Item Fields for SharePoint 2013 (Demo Project)

Some days ago I found this JavaScript library for client side encryption using standard crypto algorithms. Everything done in the browser. Cool!

 

Stanford Javascript Crypto Library: http://crypto.stanford.edu/sjcl/ – They say: “It uses the industry-standard AES algorithm at 128, 192 or 256 bits; the SHA256 hash function; the HMAC authentication code; the PBKDF2 password strengthener; and the CCM and OCB authenticated-encryption modes.”

 

Based on this I wanted to create a client side SharePoint List Item crypt module to encrypt (text) data in the users browser. So the data is stored in SharePoint encrypted. No one can read it without having the password.

Here is the code:

http://splistitemcrypt.codeplex.com/

It’s a coding exercise, nothing more!!!! Read the limitations below and be sure: there are more limitations I do not know at the moment…

 

My solution is very simple: I created a Visual Web Part with Visual Studio 2012. This web part contains everything I need for encryption / decrpytion.

1. It contains the Standford Javascript Crypto Library.

2. It contains a copy of jQuery 1.10.1.

3. It contains a Base64 serialized image that is used to mark input fields as “encryption protected”.

4. Some custom javascript.

That’s it. Small footprint. – The web part needs to be placed on each list form (new / edit / display) and on each list view page. Everything else is done by the Web Part.

 

Benefits:

  • Client side data encryption.
  • Industry standard encryption. Theoretically possible to decrypt the data later outside of SharePoint using the correct password and some tools / libraries.
  • You can share the password with anyone who needs to decrypt the data. It’s not bound to your user account.

 

Limitations – be careful to read and understand them before using it in any way. – There are more limitations. The list is not complete!!!

  • First of all: It’s a single-person’n’quick-done demo project. Nothing for production use. – You could use your data! – I’m responsible for any problems.
  • Works only for text fields and multi line text fields without HTML formatting.
  • (Single line) text fields in SharePoint are limited to 255 characters length. The encrypted data is stored as Base64 in the field. So it’s not possible to encrypt 255 characters to the same amount of data: 255 bytes of plain text chars are much more that 255 bytes in encrypted state. SharePoint and my module does not handle this situation. (Because it’s a demo project not a product 😉 )
  • If you loose the password there is no way (other than “brute force”) to get your data back. There is no back door.
  • The data cannot be searched. – You should exclude the list from being crawled.
  • No way to change the password. – If this will be possible in the future than there will be no way to migrate already encrypted data. This is because it’s client side encryption. The server does never now the password. So it cannot migrate the data from the current version of the project to a new version.
  • No inline edit on list view pages!
  • No “decrypt” option to permanently remove encryption.

At all: The project is not perfect, in any way.

 

Here is a step-by-step guide of how to use it:

1. Create a site collection.

2. Add a custom list called “Crypted Data”.

3. Add a new multi line text column as “plain text”

image

4. Click “New item”. This is the default “New” form.

 image

5. To add the crypto web part choose “Edit Page” from the site actions menu. Click “Add a Web Part”. Select category “Custom”, select web part “ikarstein’s List Item Crypt” and zone “Main”. Then click button “Add”.

image

 

6. Now you get a form section on the form asking for your password, because the password cache of your browser is empty.

SNAGHTML2b6e0a

7. After you enter the password and click the button “Set Password” you see lock icons behind the two text fields. These are added dynamically by the crypto web part.

image

8. Now you can add data as normal to the list item:

SNAGHTML2dcf6a

9. On saving the list item you will see the encrypted content for a short time. Before the encrypted and Base64 encoded data you see a prefix @@*[ – This I use to identify encrypted values.

image

10. After saving and back on the list view page you see this:

image

After adding the web part to the list view page you see this (automatically):

image

the values are decrypted using the browser cached password.

11. The same for edit form and display form. If you do not add the web part you will see the encrypted values. Like here:

image

After editing the form page you see the decrypted values:

image

The same for edit…

image

After you edit the web part you can edit the decrypted values and change them:

image

Some changes and saving it. This is the list view afterwards:

image

12. The inline edit mode does not work!!!

SNAGHTML3b7b99

 

It’s very simple. Look at the code on CodePlex. Just one note: The web party loads its own jQuery version only if the page does not contain jQuery already.

SharePoint 2013 List Forms: New Small Framework Project to Store List Forms in SharePoint Library (Part 1)

I spend some spare time in the last weeks and the last days (in Redmond before the MCSM: SharePoint rotation starts in a few hours 😉 ) for a new private development project. It focuses on custom list forms for SharePoint 2013. – Another article of my blog is one of the most recognized one regarding to the statistics: https://blog.kenaro.com/2010/12/29/walkthrough-create-custom-sharepoint-2010-list-form-for-deployment-in-a-visual-studio-2010-project/ – So I decided to write another article in this context in connection with another plan of mine to create a small framework to enable custom list forms that are stored directly in a SharePoint document library and which are connected with the content types automatically.

I release the source code there:

http://splistforms.codeplex.com

There is no “binary distribution”, because it’s “alpha“. – You will use it at your own risk – as always. Remember: It’s a private development project and I publish it for demonstration purpose only!!

 

These are the goals:

  1. Create a SharePoint list where list forms for content types are registered. – When creating a new form in this list it gets automatically registered on the content type.
  2. The list form has 3 different form modes: New, Edit, Display. All can be set individually.
  3. The list forms are rendered by using controls stored in a SharePoint document library. – The default SharePoint “Rendering Template” mechanism is used.
  4. The custom list form control can be edited with SharePoint Designer or by using another Editor by opening the control file in a mapped network drive.

 

All this based on SharePoint rendering templates. There are lot of articles out there that cover that topic.

Some information here:

Each content type in SharePoint has some properties related to “form templates” that can be set using C# or the declarative way.

Let’s have a look into the settings of the content type “Item” of a simple custom list.

Here you see all form templates are set to “ListForm”. That is the default setting. – Other lists such as task lists use different default settings like this:

 

“ListForm” and “TaskForm” are so called “rendering templates” that is taken from a certain file in the 15 hive: <15>\TEMPLATE\CONTROLTEMPLATES\DefaultTemplates.ascx

Looking into the file:

Here you find the rendering template “ListForm”. It defines how to create the forms HTML codes at rendering time. – Look at the “ListFieldIterator” tag. It will generate HTML code for all the fields in a SharePoint list. It’s very generic!! (The whole rendering mechanism is one of my favorite peace of code of SharePoint J )

Also for rendering the fields (“columns”) there are rendering templates that will be used by the “ListFieldIterator” at some point of rendering. For example:

You can create your own rendering templates for use with content types. Just add a .ASCX file to the CONTROLTEMPLATES folder of the 15 hive that contains a rendering template in this fashion. (You cannot use a sub folder below CONTROLTEMPLATES!!) Then set the property of the content type to the rendering templates name and it will be used to render your list form. – This is what my framework does.

 

In the first article I want to show you the result of this project.

In the second article [coming soon] I want to show you how it works behind the scene of the framework.

In the third article [coming soon] I want to show you how to create a SharePoint project to deploy custom list forms that uses the framework.

 

Let’s start…

 

1. To start the demonstration I need to create a simple custom list with some columns. I name the list “Simple Demo List” and create 4 columns:

  • “Date” => date and time column
  • “MultiText” => multiple lines of text column
  • “User” => person or group column
  • “YesNo” => Yes/No column

 

2. This is how the default new form of the list looks like:

 

3. Now I want to create my own list form for this list using my brand new framework. – Therefore I want to show you the two lists and one library it consists of:

  • “List Forms” : This list contains all custom form registrations made by the framework. Here you can create new forms. An event handler will create the controls derived from the chosen templates register them on the content types.

     

 

  • “List Form Controls” : This library contains the list form controls, the templates and additional jQuery libraries.

     

 

  • “List Form Templates” : In this list you can register new templates that will be used to create new forms.

     

     

    This is a simple list that refers to the “Templates” folder of the “List Form Controls” library:

     

 

4. Now I want to create a “New Item” form my “Simple Demo List”. Therefore I add a new item to the “List Forms” list:

Here you enter all information needed to identify the content type for which you want to create form. You can choose a form template. I have included to kinds of templates:

  1. Plain templates that use the same rendering template as the SharePoint default “ListForm” template.
  2. jQuery enabled form templates that allow you to use jQuery for customizations.

I choose “New jQuery” as template.

Before we hit “Save” let’s have a look at the content type using PowerShell. (It’s the same screenshot as above in the “rendering template” excursion at the beginning of this article…)

All forms are set to their defaults: “ListForm”

Now I hit “Save” on the new “List Forms” list item to create my own form.

After that the list view of “List Forms” looks like this:

What happens here:

  1. The chosen template file is taken from the “Templates” folder of the “List Form Controls” library. A new name is created using the naming template “SimpleDemoList-New-<GUID>”. The new control is saved using this name in the root folder of the “List Form Controls” library.
  2. Inside the new control file there is a template for the list columns. This template will be taken and reproduced for each (visible) field (list column). Then the template in the new control file is replaced by the list column entries. – Here is the template file:

    The whole template between the marks “LISTFORM_FIELDPLACEHOLDER_BEGIN” and “LISTFORM_FIELDPLACEHOLDER_END” will be instantiated for each list column. Inside all marked placeholders (“LISTFORM_FIELDINTERNALNAME”) will be replaced by the list columns internal name.

  3. Than a new rendering template is registered on the list content type “Item” on the “Simple Demo List”. – Let’s look into it using PowerShell again:

    Now the “NewFormTemplateName” is “ListFormNew379E”. This is the indicator that my framework will be responsible for rendering the list form.

  4. Let’s have a look into the “List form Controls” library. Here you will find the new control:

  5. I open the control using a mapped network drive:

    In it you see HTML code sections for each list column. These entries are generated automatically using the template file and the “item template” inside the template file.

5. Now let’s use the new form. Just open the list view of “Simple Demo List” and hit “New Item”.

This is our jQuery enabled form!! It works in an instant!!

The text “jQuery loaded” is generated at runtime when jQuery could be loaded. (This can be removed of course.) – This framework supports loading jQuery from library “List Form Controls” if it’s not present though the page, e.g. the master page. It also supports loading additional libraries.

To support jQuery and to allow you to use if for your own purpose there are two controls in the control file:

The first one (“LoadJQ”) is responsible for loading jQuery and additional libarires. (One library per line in “AdditionalLibraries”. Supported placeholders are: “~site” for the server relative site collection URL and “~web” for the server relative web URL.)

The second one (“JQSupporter”) can be used to store own code. As an example I’ve replaced the default “PreSaveItem” handler of SharePoint with my own handler in order to support my own validation mechanism before saving the item to be able to cancel the save action.

6. Now I want to add some new functionality to my control. Let’s say we want to ensure that title has at least 10 characters.

Therefore I modify the JQSupporter control in the control file using Expression Web 4 on the mapped network drive.

The “$$” I’ve used there will be replaced at rendering time with the ASP.NET ClientID of the List Form. This should be used to prevent naming issues. (To use “$$” as characters inside the script just use “\$$” as escape sequence.)

The variable “listform$$_ctrl_Title” is generated automatically by the JQSupporter control. For each visible list form control you will get such a variable “listform$$_ctrl_<FieldInternalName>“.

This is how the source looks like at client side:

   

Let’s test it:

It works. J

 

7. Now you are able to customize forms very quickly. “Normal” users only need read permissions on the lists/library “List Forms”, “List Form Controls”, “List Form Templates”.

All settings should be part of a normal SharePoint backup, e.g. Backup-SPSite. I’m not sure it would be part of an SharePoint Export. … I checked the exports Manifest.xml file and I can see the content types form setting is exported. Because of all controls and settings are in SharePoint lists and libraries it should be possible to include this in a SharePoint Export/Import or deployment process. – It should be possible too to save the settings in a site template. (However… you always need to add the SharePoint solution as full trust solution to your farm.)

 

Limitations:

  • It’s alpha!!!!!!!!!!!!!!!!!!!!!!
  • It is hardly tested!!!!!!!!!!!!!
  • It does not support removing the custom form from the content type. – That can be done through PowerShell by setting the form property back to the default value (“ListForm” or “TaskForm” or what ever…).
  • It should work with site content types but I’ve not tested it yet. I have commented out it.
  • It does not handle errors if they occure.
  • It’s not performance tested!
  • It has no management interface. Only the lists and libraries I’ve described above. They should be hidden. You could hide them easily.

Next steps:

  • Management interface
  • “Remove List Form” support for event handler on list “List Forms”
  • Field Rendering Templates.

Please feel free to make comments if you like it – or not like it.

InfoPath error & solution: InfoPath cannot open the following form … The file is not a valid XML document.

Yesterday I had a problem on a new SharePoint 2013 farm that took me some hours to solve it.

Problem:

I was not able to customize forms (again) using InfoPath on lists after creating them before sucessfully.

 

Steps to reproduce:

First I created a site collection and a custom list in it.

image

Than I use the “Customize Form”  button in the Ribbon menu. The new form opens in InfoPath designer.

image

Than I modify the form and publish it.

image

In the  browser I verify it’s working.

image

Than I want to change the form and click the Ribbon button “Customize Form” again.

image

Now this happens:

image

Error Message:

InfoPath cannot open the following form: http://sharepoint.local/sites/test/Lists/My New List/Item/template.xsn
The file is not a valid XML document.
DTD is prohibited.
Line 1, Position 9

When opening the URL of the InfoPath template in the browser I get this error from the Forms Server:

image

(Normally there would be a download dialog.)

 

Solution

It’s so simple… 😉 – After some network, log and code analytics I could figure out that you need…

… a site collection at the web application root!

Remember the first screenshot above. There I showed you the possible locations for the new site collection. As you can see the root “/” is available. This means: There is no site collection at the web app root.

image

After creating a site collection at the web application root I’m able to customize the list form with InfoPath. – After doing so:

image

Problem solved 🙂

 

My friend and colleague Guido could reproduce the problem and validate the fix. – @Guido: Thank you (and have a good week while teaching SharePoint 😉 )!!!

 

Hints

As far as I have seen there are other solutions related to the same problem:

  • Check your user has at least Contribute rights on the site.
  • Check your site is in “Local Sites” or “Intranet” zone of the Internet Explorer.