Note: This post is of my own personal opinion and is not endorsed or supported by any local, state, or federal government agency.
Maybe I’m completely numb to this, but can we get a final answer on the NARA issues, solutions, concerns, and policy related to the use of Twitter by the Government? I’ve seen a lot of conversations regarding it but have yet to see a final policy. I’ve heard that records management is not even required if it’s not going to keep your organization from functioning if that tool goes away…or something like that?
Some people say it’s like email with regards to records. But does that mean we have to archive every single tweet we make somehow? If so, tell us how to do it? Give us a Federal wide solution! What about responses…do we archive those too? How do we deal with twitter handles? Are there privacy policies that come into play then? If so, give us a Federal wide solution!
More and more agencies are using Twitter everyday. More and more agencies are using social media and social networks everyday. But what are the records issues and the matching solutions? Having a policy in place (that makes sense and is doable) is one thing, but providing a solution is also a need that must go along with that policy.
We need some clear answers and not loosely interpreted information. Where are the lines of text in existing policies that address the uniqueness of social media use by the Government? Show us exactly what needs to be done and then tell us HOW that should be done by providing us with instructions…is there a product that can be used; third-party contracted service; existing government services?
We need closure. I need closure. Is it required for Twitter? If “yes”, show us where the policy states it directly and provide the solution. If “no” then tell us why? If “don’t know” then tell us what the current status of creating such a policy is and how it could potentially affect current usage.
If the tweet is evidence that the government is “doing business” it is a record. Whether or not a tweet is a record is something that your organization will stipulate in its records management manual and policy. If the organization decides that twitter is not appropriate media for “doing business” then tweets won’t be “records.” But they are still non-record material and need to be managed under information, content, knowledge management policies. What NARA owes us is guidance about how to identify, capture, tag, and protect tweets/twitter streams so that if an organization does decide that twitter/yammer/etc are records, sufficient efforts will be taken to ensure they are managed in accordance with Federal Law.
Just because something is sent via email does not make it a record. If I receive bulk mail and mag subscriptions through the mail room here at JITC, I don’t log each of those things as correspondence. Neither do I save spam or non-project related emails.
So, your organizational leadership is responsible for defining and promulgating Records policy within the bounds of their agency policies. They will need your help as a technical expert so that they understand what the technology means and what the long term impacts may be.
Just my 2 cents.
jana
So what does “doing business” mean? That’s defined by each agency’s Records Management office? Does it mean performing a transaction involving money? Does it means simply responding to someone from an official agency Twitter account? If it’s any tweet that is made is considered “doing business” then what does a “record” look like? Do I do a yearly download of everything tweeted in the past year and then save it to a PDF and ship it off to the big freezer for storage? Something like that could be a problem because Twitter only allows you to go back through your tweet history for so long…certainly not a year if your account is heavily used…which all gets back to how do you archive tweets?
Or maybe, considering that it’s a third party site that has restrictions on how long data is kept, that records management isn’t even considered and the “records” that are kept are whatever the third-party allows access to whether it be through an API or directly available from the site? If this is all meant to deal with FOIA requests then maybe some sort of amendment to FOIA policy needs to be made in the case of third-party services (is there something like that already?)
Wow, what a perfect comment, Scott. I’d broaden it to all of Web 2.0 type stuff. Right now, the existing NARA guidance is unhelpful in the extreme. If I need to do records mgmt on wiki pages, how do I go about coming up with disposal proceedures, for instance? The page never has an end date, so what exactly do I do with a temporary record that doesn’t have an end date? (I would argue most govt wikis should probably not be considered records – same with twitter posts). Same issue with blogs and comments. If these communications aren’t part of an “essential transaction of the Agency” then they shouldn’t be considered records. NARA’s job is to give us FAR better guidance in making that initial determination.
@Noel: “essential transaction of the Agency” that’s the phrase I was thinking about. I don’t see Facebook, Twitter, etc as essential to any Agency. Although, that could be argued. But in terms of the policy, what I mean is if Twitter goes down, I don’t think any agency will cease to function. Is there something that defines what is meant by an “essential transaction of the Agency” or is it completely up to each agency?
If agency resources, aka public funds, are spent to develop a “message”, or distribute information to the public, that is then sent out through a variety of media, then those distributions are considered a part of “doing business.” The fact that one delivery method only fires 140 characters at a time does not, IMHO, make it small enough to be exempt from the requirements.
That said, I *completely* agree that the most useful thing NARA, NIST, OMB, the CIO council, or ANY other deliberative body in town could do would be to promulgate some minimum record retention values and definitions.
Scott,
Interesting discussion. Is NARA actively working to address this or is this something that is developing a groundswell of support to have addressed? Thanks
@Christopher: Well if that’s the case, then NARA needs to clearly articulate that and tell the rest of the Government the current status on social media and records management. As government communicators using Twitter, we need to know the answer to the question directly from NARA with information on how to go about recording it on a regular basis. Otherwise, how are we [agencies] expected to create records of a medium when NARA hasn’t provided the policy on whether or not it is, in fact, a record and how to preserve it.
@Denise: This question, and pertaining to ALL of social media, is continuously in discussion in a number of different venues. Right now I’m trying to figure out the answer from a knowledge standpoint for the potential use of social media tools in our own organization, as well as being able to respond to questions about why social media sites aren’t being unblocked. How to deal with records management of content on social media sites (any third-party service, really) is a question that keeps coming up and one that keeps getting pushed aside because nobody seems to have a factual answer from NARA.
If there’s anyone from NARA who’s listening…please please please…give us the factual answer, current policy, and information on future updates to that policy regarding how records management comes into play for third-party, non-gov’t sites and WHAT is the solution for creating those records. Seems like a simple question and answer, but it’s one that has yet to be answered directly.
For what it’s worth…
As I look at the archiving public records & governmental communications from Twitter, I see an opportunity for somebody to develop a Twitter client that archives the tweets & DMs from the account along with @ tweets & DMs to it. Twitter makes its API available to the developers.
Hi Scott, “Essential transaction of the Agency” is pretty much the key definition of whether something is a record or not (there are other parts to this as well). Each agency makes a determination of what they think is a record, but its up to the Chief Archivist of NARA to actually make the call. Again, the problem with the Records Mgmt policy, as with so many others is that social software really changes the nature of the game. No longer are we looking at static moments in time that relate to a very specific process – we are now talking about ongoing conversations that may relate to many separate issues. Said another way, in determining whether your tweets should be records, are they all related to the same basic level of information or larger decision making process? Of course not. Neither are all wiki pages in a site, nor all blog posts in a site. Unless the social software effort is set up for a very specific purpose related to a specific essential transaction of the agency (giving authoritative tax advice on blog posts for filling out 1099s, for instance), chances are the whole question gets very muddy very quickly.
This in a nutshell is why NARA is struggling with this. At least from the folks I’ve talked with in NARA, yes, they know the issues, but I don’t think they’ve come up with any magic answers. The other issue is figuring out records on Federal sites versus those on commercial sites like Facebook and Twitter. The Federal Govt has no control over whether Facebook deletes a post from their database or not. If this is done on a WordPress blog on a Federal webserver, you at least still have the information in the underlying database.
Here is the US Code definition of “record.”
§ 3301. Definition of records
As used in this chapter, “records” includes all books, papers, maps, photographs, machine readable materials, or other documentary materials, regardless of physical form or characteristics, made or received by an agency of the United States Government under Federal law or in connection with the transaction of public business and preserved or appropriate for preservation by that agency or its legitimate successor as evidence of the organization, functions, policies, decisions, procedures, operations, or other activities of the Government or because of the informational value of data in them. Library and museum material made or acquired and preserved solely for reference or exhibition purposes, extra copies of documents preserved only for convenience of reference, and stocks of publications and of processed documents are not included.
If you are providing the “official response” to a citizen, another branch of government or a foreign entity, you are “doing business” and need to capture not only the “official response” but the request as well. The method for receipt of the request or delivery of the response is irrelevant to the definition of “record.” (Which is what makes this all such a fun discussion.)
Scott, your organizations records policy should address the questions you asked. If it doesn’t, please bang the people responsible for those policies in your organization. If you are the responsible person, PM me and I’ll see if I can help you find the person from NARA assigned to your area. They will be glad to help. If you are DoD, I’ll introduce you to your service RM. You are asking great questions! What a great discussion!
http://www.archives.gov/records-mgmt/training/rm-everyone.html is the basic training for what is and isn’t records from a NARA perspective. IT is required annual training for all govt employees. It is mostly ignored.
NARA doesn’t have any way of knowing each organization’s mission and isn’t qualified to determine what was and was not to be kept as records until someone at the organization gives them something to go on…or they come in and do an audit. The processes for standing up new federal organizations does not include a NARA check box. (Maybe it should.)
Noel, you are so right about Records Management Policies. Have you looked at Google Wave? How are we going to identify “records” from that? How do we keep this stuff useful for hundreds of years?
We need to constantly revisit why we keep records in the first place and how we will meet those needs with constantly evolving technological artifacts.
Christopher, record retention is not determined by media. It is determined by content and purpose of the record. The organization authorizing the record creation process is the place that retention period needs to be defined. Not all tweets, not all email, not all blog posts are equal. Some are vital…most are trash. One size does not fit all.
Third party sites and cloud computing are interesting? Who owns the data?
The other day, Vivek Kundra streamed a live announcement about data.gov on Facebook. That streaming video IS a federal government record, as were all the incoming comments and questions. I don’t know if or how that interchange was captured, but it should have been.
I’m going to fall back on the canard, “It is your organization’s policy” If your organization decides to use third party methods for “doing business” with the government, it is that organization’s responsibility for coordinating with the third party the methods for capturing records and providing them to the organization so that they can be protected as federal records.
Hi All – NARA is working on it! See links below:
1. Proof that someone’s educating about Gov 2.0
2. Evidence that they are thinking about social media and records management (and a useful set of slides)
3. Smart people gathered to address this issue six weeks ago (the current Mecca for records information, methinks)
By the way, I emailed Robin Riat and Arian D. Ravanbakhsh at NARA (their slides are above) to invite them to this discussion.
– Andy
@Noel: “The Federal Govt has no control over whether Facebook deletes a post from their database or not”…that’s the exact kind of problem I’m talking about. Government doesn’t have control over third-party sites like Facebook or Twitter. So the only way to keep a record of every single action taken is to build automated tools, manually do something, etc and keeping every single interaction in a logical fashion…all of which is a huge expense on taxpayers. Government tredding into social media is already seen by some as a waste of taxpayers dollars…I don’t agree though since there are many business cases for participating. However, if we’re then required to archive third-party sites and every interaction undertaken on each then the Government will need to spend even more taxpayer $$$ to make that happen on a daily basis. That’s not efficient.
If there is some record that is needed then there needs to be some criteria, beyond what is already written, that defines what constitutes a record on third-party sites. Maybe it’s a “snapshot” of currently available information once every 6 months, or once a year. Maybe NARA should establish a Federal contract with Archive.org (who already have the infrastructure and process in place) to build a Federal archive of content specifically to deal with all records retention issues on publicly facing government and third-party sites. Who knows.
But if NARA isn’t able to provide a better understanding of what is and isn’t a record, and how to archive it, then I don’t see how it should further act as a showstopper for agencies to participate in social media. How are other organizations doing it? There’s a big federal presence in social media sites…what are they doing? We should probably be aware that records retention, in some form, will have to happen at some point in time but for now continue operating as is.
@jana: “appropriate for preservation” is where I see this catch as was eluded to earlier. If the agency says “We don’t feel it’s appropriate, or needed, to retain records of tweets or interactions on Facebook” then that’s that, right? If it’s being left up to the organization to the make the final decision…and that decision is relayed back to NARA with justification…then shouldn’t serve as the base for all other records retention decisions by the Federal Government?
Has any agency given NARA some justification for not retaing records from a third-party site?
Hi Scott
About your comment . . . “essential transaction of the Agency” . . . you are describing a “vital records” program. That’s one aspect of the full records management program where agencies determine what records need to be accessed in order to function in the event of a crisis or emergency. Think natural and man-made disasters. Some social media sites have proven very useful during crisis situations because they are third-party, not affected by the disaster, and can be easily accessed. Think of when your agency webservers go down, but you can still post tweets/fb updates about actions/activities via your mobile to your agency’s Twitter account.
As for NARA, I don’t think we can expect step-by-step instruction on how to save Tweets from NARA. I don’t think I want that as NARA probably can’t set the policy as fast as the technology moves. As for what’s a “record tweet”, I think tweets sent on behalf of an organization or official in my agency must be managed the same as other Federal records. That means getting the record tweets into an electronic recordkeeping system. But I think that tweets/FB posts sent by individuals do not count as Federal records and instead fall under the personal papers category. I think that when in doubt, err on the side of caution and save the tweet. For example, is this post from Lisa Haralampus, the RM rock star, versus Lisa Haralampus, the program manager at State? (I hope its purely personal and not something my agency needs to keep.)
I’m considering using RSS feeds to capture the social media content in a gov-owned set of servers, assuming that the gov-provided content (a.k.a. records) won’t be available via the third party providers. I see some wonderful opportunities here for Federal cloud computing and harvesting/managing social media data. While individual posts/comments do not have value, the concatonated whole offers a very unique insight into how government works and how goverment employees accomplish their missions.
So you’re going to be building a method for archiving changes in an RSS feed and then storing that entire archive on government servers, correct? So, that means you need to then address the privacy issues since archiving means you’re also grabbing Twitter handles as well as their interactions with you. And since many tweets have geographic location tied to them then there’s the potential that you could track them back to their location, or locations. Plus since you’re retaining records and interactions then you’ll need to address for how long you’re going to keep those records..which of course the answer is indefinitely since you’ll be sending them to NARA for storage every “x” years based on your Agency’s schedule.
I’m just wondering how NARA and Agency policies are going to apply to the changing world of social media and social interaction between government and public. I haven’t yet read Andrew’s documents above, but looking forward to doing that.
I’m reading through this NARA page (http://www.archives.gov/records-mgmt/initiatives/web-tech.html) as referenced from Andrew’s PDF fiile above titled: Implications of Recent Web Technologies for NARA Web Guidance.
Reading this information, you can see that it clearly applies to a government entity hosting some sort of web 2.0 tool (wiki, blog, RSS, portal) on government hardware which may, or may not, allow collaborative and frequently changing content as well as pulling in of external non-gov’t content. But this does not address government participating in a non-gov’t wiki, blog, portal.
If content placed into non-govt sites is still considered to be part of a Federal record, then according to the current guidance by NARA it would seem to be an undue burden placed upon agencies to collect and manage records from a site such as Twitter, Facebook that are changing by the second as opposed to static content. If something like that was on government hardware then there are database backups and such, but when it’s a site not under our control how can we effectively, and without wasting millions of taxpayer dollars, continuously capture content?
How come nobody is suggesting the US Code definition of “record” that Jana extracts be amended?
Ari, that’s a good point and probably one major place that should be changed. How would you amend it though?
As used in this chapter, “records” includes all books, papers, maps, photographs, machine readable materials, or other documentary materials, regardless of physical form or characteristics, made or received by an agency of the United States Government under Federal law or in connection with the transaction of public business and preserved or appropriate for preservation by that agency or its legitimate successor as evidence of the organization, functions, policies, decisions, procedures, operations, or other activities of the Government or because of the informational value of data in them. Library and museum material made or acquired and preserved solely for reference or exhibition purposes, extra copies of documents preserved only for convenience of reference, and stocks of publications and of processed documents are not included.
I know this isn’t going to happen overnight, but it’s apparent that some sort of rewrite is needed to the NARA regs regarding what constitutes a record in light of 3rd party services and gov’t engagement. However, knowing that won’t happen now…and an answer is needed now…I suggest that NARA at least put out some sort of “interim” language that can be used to at least give agencies something to go with for now but that doesn’t prevent them for participating or spending hours of time figuring out how to archive content that changes every second.
I’d also like to suggest NARA open up dialogue to the public (gov’t and interested third-party organizations) to come up with new/amended regulations that meet the needs of both the Federal Government in terms of records management and not creating an undue burden, as well as any issues that third party sites may have.
This may be naive, but why can’t twitter conversations be considered akin to phone conversations and the like?
Good point, Teri. I’ll go further. Wouldn’t a telephone conversation be a “machine readable material” in accordance with the above code? Oughtn’t a phone chat be marked for archival?
I believe phone records ARE logged and archived as well…if I’m not mistaken.
Andy – Thanks for the heads up and pointer to this discussion. I’ll add a couple of thoughts with the big caveat that these are my opinions and should not be read in any way as constituting official NARA policy.
First, the CFR is clear that each individual agency is responsible for identifying their records. NARA’s role is to provide agencies with an understanding of what to do regarding records management (and why they have to do it) as opposed to how those individual RM requirements get carried out. Once information is determined to be a record, it may not be disposed of without the consent of the Archivist of the US (via a SF-115 and the standard NARA appraisal practice).
The notion that social media/web 2.0 change things is accurate to a point. I think the guidance spelled out in the Managing Web Records guidance is still on point. I’ll also mention several other pieces of NARA guidance that have been developed that would be relevant to this discussion.
1) If you equate Twitter postings with IM – not a far stretch – then our IM FAQ might be relevant:
http://www.archives.gov/records-mgmt/initiatives/im-faq.html
2) We’ve also developed guidance on Selecting Sustainable Formats that advises agencies on how to choose electronic formats for records:
http://www.archives.gov/records-mgmt/initiatives/sustainable-faq.html
These and other resources and best practices in records management are available in our Toolkit for Managing Electronic Records at: http://toolkit.archives.gov
One final thought – I know that NARA is certainly aware of these issues and is working to provide agencies with the right guidance and tools to meet their business needs.
Arian
Arian,
Thanks for that great information. I read through the IM information and I do see where Twitter is easily equated to instant messaging. However, my concern is that it still seems to be more pointed toward applications, or services, running ON agency networks:
That leads to the question, “If I’m using Twitter at work to post official tweets, is that considered allowing “IM traffic on” my agency network? Maybe I’m getting into the weeds here, but I think there needs to be some clarification there. Now if using Twitter IS considered “IM” then A) that document needs to be updated at some point and B) we will need to have some examples by other agencies on how they “log” IM traffic.
But if the tweet is not considered “IM” but instead a standard “web record” then the question is, “Do interactions on third-party sites not government owned constitute a web record?” I know that language says “machine readable materials” and “made or received by an agency of the United States Government” but if it includes third party sites then the language needs to specifically state that. Right now it seems like such a gray area.
In the end, if I’m not mistaken, it says that we need to work with our Agency reps to determine if “tweets” or Facebook posts, etc are considered valid records to be captured. If the Agency says “no” then they need to ask a NARA official who gets the final word? If “yes” then there are bigger issues to be concerned with regarding privacy, personally identifiable information, resources and methods for capturing content, etc, correct?
To further Scott’s question, here’s another for Arian:
If a government employee posts a comment on youtube.com after watching a video, is that comment required to be archived? If so, how to prove the comment was made if the commenter doesn’t report it?
If a gov’t employee posts a comment on GovLoop does it have to be archived? No, I don’t believe so. However if that person is responding in an official business capacity then maybe.
OK – again, these answers are worth the price you paid…
If the agency determines that a Tweet or a posting to Facebook is a record, then it must make the effort to capture that record for as long as it is needed for agency (and perhaps NARA’s if it appraised as a permanent record) purposes. Which means they would have to get a records retention schedule (SF 115) approved by NARA for the disposition of that material. NARA would be agnostic on how that is accomplished. For my personal use, I’ve been using Tweetake to capture and retain all of my Tweets in .csv format. It’s a nice solution for me, don’t know how well it would scale up.
To Ari’s example of making a comment, I think Scott is correct. If it’s clearly personal, not a record. On the other hand, if in an official capacity it may certainly rise to the level of the record and have to be preserved.
To me, this discussion simply underscores what I’ve been saying to my professional colleagues (some at NARA, most within the larger archival/RM community) – that the traditional concepts of the 20th century records management world are evolving. We’re thinking about all of this – we just haven’t articulated the answers yet.
To make this easier and cheaper for government it would be nice if various Twitter clients like Seesmic Desktop, TweetDeck, and others would provide an option that would automatically keep an XML or CSV type file of all sent tweets from any selected account (if you manage multiple accounts), including direct messages. This way we could simply transfer the log file to NARA for official government Twitter accounts based upon our disposition schedules.
This would also remove the issue of having to worry about whether we place an undue burden on organizations to capture tweets as records, or not, since the tools would perform this function automatically.
A next step would be to have a .gov entity, or have NARA or GSA, partner with a third-party that could provide a web-based tool for allowing the same capability of automatically capturing sent tweets and DMs from authenticated government Twitter accounts.
GSA? NARA? Anyone? Who at your agencies can make this happen?
It is really good to see people talking about records. Like Arian, I’ll add my (personal, not in any way NARA policy) 2 cents.
As others have already said far better than I could, the content, purpose, and use – not the format – of the “recorded information” (e-mail, document, blog post, IM, recorded phone call, tweet, wiki, etc.) are what make it a record. That’s why it’s not easy to declare a format to be a record/nonrecord. Is a photograph a record? Is a paper document a record? It depends on who’s creating/receiving/using it, and for what purpose. I really do wish I could give you an easy, one-size-fits-all answer. On the other hand, if NARA did that, I promise you’d hate it because one size never really fits all.
I think the good news here is that making that call doesn’t have to be painful. If your agency is starting a new Twitter feed (or a new insert-next-social-media-thing-here), someone is probably considering how it will be used and how it fits into the other work the agency is doing. You just take one additional moment to think about the value of of that to the agency, its customers, and to history. When it comes to deciding the retention period, your agency may already have a records schedule that applies. For example, if my agency has a grants program, and we already have a records retention period for records related to that program, it might apply to our Twitter feed for the program, too. If it doesn’t, it’s not that hard to make the decision and get the schedule approved.
It would absolutely be easier with the right tools to automate the process. What the records management world has seen over the last several years (and last several new technologies) is that the technology develops much faster than the tools and policies available to manage it. With Twitter, it seems like the right people working with the Twitter API could at least make a dent in the problem.
As Arian said, the records/archives world is thinking about this. The traditional concepts of record, retention, and disposition (preserving something forever in an archives or deleting/destroying it at a specified time) really are evolving. Managing government information intelligently is always going to be a good idea, but exactly how that happens and what that means has to change over time. The trick is to get the social media, IT, records, and archives communities working together.
Thanks for your perspective Robin. I’ve been talking with another group in our organization that has built a Twitter-archival tool for a new project that they’re prototyping. In discussions with them it sounds as if you can record “sent” tweets for up to 200 accounts based upon the API that Twitter offers. If any agency has more than 200 Twitter accounts then I think there’s a communications management issue there. Then it’s a matter of dumping it down to a “sustainable” format such as XML or CSV. Apparently, they’re doing this now in some fashion and with a little tweeking could make it work. So, I will certainly continue looking at that option. Right now the focus is on the main project they’re doing, but it’s certainly something that is on the plate. If all works out well, and they’re OK with the tool, I’ll see if they’d be willing to let me share that with everyone. It would certainly be of benefit. Not sure when that will be…but I won’t forget.
Very interesting discussion! I am in government at the state level, and we share the same concerns. Two thoughts I haven’t yet seen in this string: FIRST — Once things make their way to the Internet, they seem to take on a perpetual life of their own (and most are undated, which makes getting the most current information a tremendous challenge). If these things truly have a life of their own in cyberland, does it make sense for a government entity to have responsibility to archive them also? SECOND — Since all of Web 2.0 (or 3.0 as some indicate) is in constant Beta testing, meaning that a new technology has replaced the old before the bugs were worked out of the original, would everything thus be considered a “Work in Progress” and as such exempt from open records/open government?