Houston…we have a problem with reliability
A note from Mark Miller: Long time reader, Magnus Rygge, has a problem: “We feel very insecure of how we should treat our documentation to keep it reliable for back tracking and knowing what was changed, by who and when.” Here’s his complete problem statement. Hopefully, you can give him some help and guidelines.
1 Background
We are a small company acting on a long time regular consultancy basis for a number of companies. We develop manage and store documentation for these companies. The documentation created is sometimes only for one single company and sometimes complete copies of the documentation should be distributed to several of the companies.
2 Environment
- SQL 2005
- SMB 2003
- WSS 3.0
- Settings applied:
– Check out required for editing (the users “My documents/ SharePoint drafts” folder is on a server not at the C:)
– Major and minor versioning - Storing of metadata is both in site columns and in document library columns. The document library columns are equally named and created through the same document library template. In the beginning we didn’t understand the strengths of site columns…
- No programmed customizations
- No work flows implemented
- Only used for intranet purposes
- No connections to MS Outlook
- SharePoint is mainly used for storing and sharing documents, very few list types other then document libraries are used.
- Start up of the installation about 1 year ago
- SharePoint Designer 2007 SP1
- XP Professional
- Office 2007
– In headers of documents (inserted with command Quick parts) metadata is used
– Our documentation includes both Office97-2003 and Office 2007 documents. - Approximately 4000 documents within the installation today and will increase
3 Main issue
We want to be able to do the following without losing any document information/ metadata:
- Rename a “parent” folder
- Move a folder or document to another document library
- Move a folder or document to another site within a site collection (to another site collection would be good but is not as limiting)
After a lot of testing to see how SharePoint reacts to these actions we feel very insecure of how we should treat our documentation to keep it reliable for back tracking and knowing what was changed, by who and when! Searching the web we can’t find much about these issues. We couldn’t be alone having this as a requirement for a document management system or are we?
Below I try to describe in more detail how we would like SharePoint to “behave” and how SharePoint “behaves” for us now. Screen shots from tests performed to describe our issue are included. For integrity and security reasons we have spray marked sensitive information.
Since the start up of the installation we have found indications that there is something fishy about Modified, Modified by and Version columns but haven’t been able to find a repeating pattern. Now, with the documented tests below, we seek support to trouble shoot our issues and to find solutions or indications on how to proceed.
4 Important Metadata to be Retained with Documents
To be able to back track the development of our documentation and to see who did what and when we need to retain, at least, the following information/ metadata:
- Modified
- Modified by
- Check in comments
- Custom site columns and columns in document libraries (we know there are limitations in metadata functionality for Office 97-2003 documents)
- Version history for documents, including all above bullets
5 Renaming Folders
As a result of testing we have found that depending of the degree of renaming, the effects on the folder content are different! I cannot guarantee that this is valid for all situations but this supports my “bad” feeling that SharePoint is not consequent. The pictures below are screen shots from tests performed on 6/25/2008. Two site collections have been used, Test 1: SharePoint and Test 2: Management. The reason for this is that the information at the Management site collection was less valuable.
5.1 Test 1
In the test below SharePoint is the top level and Documents is a document library.
5.1.1“Minor” renaming
In example below only “Test” is added at the end of the folder name “General administration”:

Figure 1 Before renaming.

Figure 2 Inside folder “General administration” before renaming.

Figure 3 After renaming folder “General administration” to “General administration Test”,
Modified, date and time, of folder “General administration Test” is changed, ok.

Figure 4 Inside folder “General administration Test”, nothing inside folder changed, ok.
5.1.2 “Major” renaming
In example below “General administration Test” is renamed to “General Test administration Test”:

Figure 5 After renaming folder to “General Test administration Test”, Modified is updated to 10:57, ok.

Figure 6 Inside folder “General Test administration Test”, Modified has changed to 10:55 (why is there a difference in time, it didn’t take 2 min to process the change…) but Modified by is not changed, not ok.
5.2 Test 2
In the test below Management is the top level and Magnus is a document library.
5.2.1 “Minor” renaming
In example below only “Test” is added at the end of the folder name “SharePoint” (this test was performed before Test 1):

Figure 7 Before renaming

Figure 8 Inside folder “SharePoint” before renaming

Figure 9 After renaming folder “SharePoint” to “SharePoint Test”, Modified of folder “SharePoint Test” is changed to 10:50, ok.

Figure 10 Inside folder “General administration Test”, nothing is changed, ok.
5.2.2 “Major” renaming
In example below “SharePoint Test” is renamed to “Share Point Test” (a space added), this test was performed after Test 1:

Figure 11 After renaming folder to “Share Point Test”, Modified is updated to 11:30, ok.

Figure 12 Inside folder “Share Point Test”, Modified has changed to 10:50!!! but this was the time for the first rename action! Not ok.
5.3 Summary…
- Renaming a folder changes the Modified date of the folder itself and can change all files and folders down the folder hierarchy. Modified by however remains unaffected. This is very tricky as you can find documents being changed at dates by persons not being contracted at that time.
- Renaming document libraries does not affect anything (as far as I have discovered).
6 Moving items
6.1 Within a site
Moving items (cut and paste documents or folders, performed from Open with Windows Explorer window activated from the Actions menu) between document libraries within a site we thought worked fine. But initial tests were performed without metadata entries and we only checked the version number and saw that old versions were retained. Only occasionally we found inconsistencies… But upon performing the below test it is clear that moving between document libraries is not working as we would like it to!
6.11 Test 1 moving within a document library
A word 2007 document (.docx), spray marked red below, is moved within a document library “General subject projects” to a subfolder “spray marked green”. Test performed 6/27/08.

Figure 13 Before move of red marked document to green marked folder.

Figure 14 Version history of red marked document before move.

Figure 15 Target, green marked, folder before move.

Figure 16 Target, green marked, folder after move of red marked document. Modified, Modified By and metadata retained, OK.

Figure 17 Version history for red marked document after move. Metadata entries are retained to the correct versions! This is ok.
Test 2 moving between document libraries
A word 2007 document (.docx) spray marked red is moved from document library “General subject projects” to a subfolder “spray marked green” in document library “Company_related_projects”. Test performed 6/27/08.

Figure 18 Before move of red marked document, document last changed 10:31 (a change of the comments column, a customized column not equal to check in comment…).

Figure 19 Version history of red marked document before move.

Figure 20 Target folder, marked green, before move.

Figure 21 Target folder, marked green, after move of red marked document. Modified changed to 10:34, acceptable but not ideal.

Figure 22 Version history of red marked folder after move. Modified by is changed for all versions and metadata input is grouped to the last version!!! Not ok!
6.1.3 Summary
We would like SharePoint to handle all move actions as it does for moving within a document library, ref 6.1.1.
Issue with metadata being grouped to latest version as shown in Figure 22 is very strange, for example Title and Document number are used in some Word document headers (inserted via Quick parts). According to the version history only the latest version of the red marked document should have that metadata but when opening older versions the information is there!
6.2 Between sites within a site collection
Moving items (cut and paste documents or folders) to a document library in another site (within or between site collections) reduces the information further. No version history is retained and Modified by and Modified is changed to the person who moved and to the date/ time the action is performed. However metadata is retained for the single version moved.
6.3 Export/ Import with SharePoint Designer
As we understand a method that should work for moving document libraries, folders or single documents is to use SharePoint Designer and export/ import. When testing; metadata information is retained but version history is lost, Modified by and Modified date will be the person importing and the time for the import. In short this is the same result as is for cut and paste, shouldn’t it perform “better”. We tried to remove the require checkout setting but with no change in result. Are we right in thinking that export/ import should work?
7 Bottom line
We do not know how to continue. Is SharePoint WSS 3.0 the correct tool for us or should we upgrade to MOSS (or is the functionality regarding move and rename for MOSS equal?)? Could we have missed something during our set up of the installation and therefore have SharePoint not working properly?
Guest Author: Magnus Rygge
We have struggled with moving documents between libraries and sites here. One thing I’ve noticed is that I will keep version history using cut and paste (not copy and paste) as long as the proper versioning is turned on in the new library.
Obviously this only works for libraries within the same site collection.
Another option, that I’ve occasionally found problematic, is under Site Actions, Site Administration –> Site Content and Structure. Documents can be moved through here and retain their information I believe. I have had this create duplicate columns in the new library though, as well as break the modified column.
There are also 3rd party tools for moving documents.
I almost forgot. The other thing I do whenever moving documents is copy all metadata to Excel. Assuming it doesn’t copy over properly I will just paste the metadata in with required check out turned off.
This works well if you use Edit in Data Sheet.
Very interesting post. I expect more features here in WSS 4.0 around MDM (metadata management).
I use http://www.codeplex.com/SPDeploymentWizard for moving items around with full preservation of metadata tags and version history. It even works across farms (dev\test\prod). Server side tool only but the code is open source … so … a little C# code creativity and maybe you could expose this as a Feature to end user in the GUI?? Something to research.
Hi,
Thank’s for the respons upon my posts. I think that Office 2007 documents helps you to maintain metadata when moving documents around. Metadata is embedded in the new x-files, try to download a document to your computer and upload it again, at least my metadata is left unaffected so there is no ShP help necessary, Office takes care of that. We use Quick parts to enter metadata (like doc number) into word-documents and that works fine evan off line.
My problem is the document versions and their Modified and Modified by metadata, that is only maintained if you move within a doc lib.
SPDeploymentWizard, I tried it some but I can’t move single files! You can move single files but then they must be exported and imported together with their parent doc lib, but that doesn’t solve my problem, I want to move documents between doc libs!
Jeff did you really manage to move single files between doc libs?
I had a programmer to check this out but he has now stopped, he managed to move documents and their old versions bewtween libraries but he couldn’t maintain the Modified and Modified by fields after the move. Also found some applications to buy but these had the same limitation. Can this really be a limitation in ShP, any fresh ideas are welcome!
/Magnus
Does anyone know if the modified by and modified date issues are fixed with SP1 or SP2
I’ve found a solution for moving documents around within a SharePoint site collection; CopyMove from SharePointProducts, http://www.sharepointproducts.com.
CopyMove seems to be the ultimate solution if you want to be able to restructure your documents on the site and want to maintain verson history and metadata entries. According to my tests you can evan move a document to a target doc lib having fewer columns than the source and afterwards when adding the proper site columns they got populated with the corresponding metadata!
Convenient user interface and easy installed, can’t guarantee anything, but I’m impressed!
Regards
/Magnus