PREFACECONTEXTWisconsin Department of Public Instruction’s (DPI) WISEdata project is about improving Federal reporting. Federal reporting can be improved by replacing individual legacy collection systems (ISES, CWCS, etc) with a cohesive interoperability collection framework. DPI has chosen the Ed-Fi interoperability framework to facilitate collecting local education agency (LEA) information. This framework will need to be implemented (if it is not already) by the LEA’s Student Information System (SIS). The SIS software will push data to DPI via a web service defined by the Ed-Fi framework (see Figure-1). PURPOSEThe purpose of this document is to describe the process of sending and receiving data using the Ed-Fi REST client against the Ed-Fi API hosted by DPI (WISEdata). The API follows REST guidelines which makes sending and receiving data relatively standard (via HTTP verbs like GET and POST). This document also describes the authorization using OAuth 2 which is critical for connecting to the Ed-Fi API. Note: This information is also documented by Ed-Fi and DLP. It is strongly recommended that you review their documentation which is listed in the references sections. USE OF THIS GUIDEThis document is appropriate for programmers who wish to write clients to send requests to the WISEdata via secure RESTful Ed-Fi API.
The use of this document assumes no particular programming language or framework. However, the programmer should be familiar with RESTful services and Basic HTTP Authentication. A brief overview of these concepts is given below. References for additional reading are also provided
DEFINITIONS, ACRONYMS, AND ABBREVIATIONSTerm | Definition | CWCS | Course Work Completion - DPI legacy system - Collects relational information about students and courses. | DLP | Double Line Partner - Ed-Fi partners with DLP as the development shop for their organization. Learn More: http://www.doublelinepartners.com/ | DPI | Wisconsin Department of Public Instruction Learn more: http://dpi.wi.gov/ | Ed-Fi | Ed-Fi technology serves as the foundation for enabling interoperability among secure education data systems designed to improve student achievement and teacher satisfaction. Learn more: http://www.ed-fi.org/ | HTTPS | Hypertext Transfer Protocol over SSL (Secure Socket Layer) | ISES | Individual Student Enrollment System - DPI Legacy System - (subsystems include ISES Third Friday of September, Year End, Child Count, and Discipline) Includes student demographic and outcome data needed to meet the ESEA report card requirements at the state, district, and school levels. The ISES requires that schools and/or districts submit data on students enrolled during the previous school year, students enrolled on the Third Friday of September of the current year, students with disabilities enrolled on October First of the current year, and students removed under disciplinary action. | OAuth 2 | OAuth 2 provides client applications a secure delegated access' to server resources on behalf of a resource owner. Learn more: http://oauth.net/2/ | REST | Representational State Transfer | WISEdata | With regards to this document - WISEdata is the DPI implementation of the Ed-Fi interoperability framework to facilitate collecting local education agency (LEA) information. WISEdata has other focuses such as student privacy and security. Learn More: http://dpi.wi.gov/wisedata |
REFERENCES
SYSTEM OVERVIEWThe Ed-Fi API exposes endpoints for submitting and retrieving educational data. Those calls are secure and synchronous (the response is immediately returned). PREREQUISITES The Ed-Fi API is service secured via OAuth 2. An OAuth key and secret (per district) is needed to make REST calls. Secret Get from DPI - it will be in the Getting Started Guide
Key Get from DPI - it will be in the Getting Started Guide
Each OAuth credential needs to be associated with the correct district which will be done by DPI. Test district ID Get from DPI - it will be in the Getting Started Guide
Test school Get from DPI - it will be in the Getting Started Guide
A REST client which can authenticate via OAuth 2. There are many free and paid REST clients that you can use for testing. The two clients that we use are listed below. Both are free. SoapUI (version 4.6.4): Besides supporting SOAP requests, it also has a REST client.http://www.soapui.org/ RestClient 2: This is a FireFox extension. According to the developer, they also have an extension for Safari and for Chrome. Developer website:http://restclient.net/
A web traffic tracer / debugger. There are a number of web debuggers out there. The REST clients above will (SoapUI or RestClient2) will show web traffic when you make a request. The web debuggers can be handy when creating request via the DPI provided WISEdata Ed-Fi REST SDK or a proprietary Ed-Fi REST client. Fiddler is free and one of the more popular web debuggers http://www.telerik.com/fiddler Wireshark https://www.wireshark.org/
WISEdata Ed-Fi REST SDK.zip which is. The SDK was Written in C# / .NET and developed using Visual Studio 2013. The SDK has code for OAuth 2, Ed-Fi REST clients, and integration/unit tests. There is more information about the SDK in the Appendix - WISEdata Ed-Fi REST SDK The Ed-Fi REST clients can be created manually for C# or Java using the DLP generator (see Appendix - Manually Creating the Ed-Fi REST client). It is not mandatory to use the DLP provided client. However, when developing an Ed-Fi REST client adhere to the Design Guidelines.
AUTHENTICATION - OAUTH 2PREREQUISITES A REST client is required to make REST calls to the Ed-Fi API. The client that will publish data to the Ed-Fi API should be the SIS vendor software. The SIS implementation will likely be based on the Ed-Fi REST client (an example of the client code is listed below in Appendix). The client which is being used in the testing phase is called Swagger UI https://uaapps.dpi.wi.gov/EdFiSwagger/
A client key and secret is required to connect DPI will provide the key and secret for each LEA/District to SIS vendors. These are credentials to authenticate/authorize a client. See Figure 2 for an example. Note the key/secret pair are entered first. Next, the user has to press the “Get Token” button to get a token which is required to make REST calls.
Authorization via OAuth 2DEFINITIONSOAuth 2 This is a protocol that provides clients access to server resources without including client credentials. For more information see: http://en.wikipedia.org/wiki/OAuth For more information about Ed-Fi OAuth 2 authentication see http://www.ed-fi.org/assets/2013/11/Public-Ed-Fi-REST-API-Design-Guidelines-1.2.pdf
API URL The URL to the Ed-Fi API RESTful web service ex. https://edfi-rest.dpi.wi.gov/ (not a working URL)
Admin URL The URL to the Ed-Fi OAuth web service https://uaapps.dpi.wi.gov/EdFiWebApi
Key Part of the clients credentials DPI will provide each LEA a Key ex. ekVNpbjj6SbQ
Secret Part of the clients credentials DPI will provide each LEA a secret ex. 9mnMlZKaswgW9RizPdEKdk
Token A value obtained in the OAuth protocol which is used to access the API Server
Ed-Fi Two-Legged OAuth 2 Protocol Sequence:POST https://uaapps.dpi.wi.gov/EdFiWebApi/oauth/authorize Include the paremeter “Client_id” with the clients Key value include the paremeter “Request_type” as “code” Store the response as client access code
POST https://uaapps.dpi.wi.gov/EdFiWebApi/oauth/token include the parameter “Client_id” with the clients Key value include the parameter “Client_secret” with the clients secret value include the parameter “Code” with the clients access code value include the parameter “Grant_type” with the value “authorization_code” Store the response as the client access token
Make REST API calls to https://uaapps.dpi.wi.gov/EdFiWebApi/api/v2.0 Include a header “Authorization” with the value “Bearer” + Token
SERVICESAll of the endpoints, sample data, and schema information can be found and tested via Swagger UI located here: https://uaapps.dpi.wi.gov/EdFiSwagger/ PROFILES:The following is a document which describes profiles in the Ed-Fi API: https://techdocs.ed-fi.org/display/ODSAPI20/API+Profiles Wisconsin DPI’s main purpose with profiles is to limit the amount of data that a SIS vendor can send via the API. Wisconsin DPI only wants to collect certain elements as noted above. The default API Section dropdown value is Resources. A SIS can see the resources as restricted by the profile change the dropdown. The API requires a content type be sent with a any HTTP action (ex. GET, POST). The content type is standardized, but lengthy. Public school clients trying to GET a student have to send the content type “application/vnd.ed-fi.student.widpi-profile-public.readable+json” which is highlighted in the image below. If the same SIS wants to POST a student school association they have to send the content type “application/vnd.ed-fi.studentschoolassociation.widpi-profile-public.writable+json”.
Note: the SIS still send the JSON to the same endpoints regardless of profiles. CLAIMSETS:The following document has more detail on claimsets: https://techdocs.ed-fi.org/display/ODSAPI20/Security+Configuration+Data+Stores Wisconsin DPI restrict resources in the Ed-Fi API by linking a SIS key to a claimset. The claimset is linked to resources that a vendor can access and how they access the resource (authorization strategy). SIS vendors don’t need to know a lot about claimsets. If you GET/POST a resources outside the Wisconsin DPI domain you might get a error about a claimset. Wisconsin DPI can help confirm if the issue. SAMPLESAll of the endpoints, sample data, and schema information can be found and tested via Swagger UI located here: https://uaapps.dpi.wi.gov/EdFiSwagger/ Note that samples may not contain all data required or conditionally required by Wisconsin DPI, but the samples should help you get started. You may find JSON samples in the Data Domains & Endpoints area under each endpoint. A couple of key notes: studentUniqueId is the same as WISEid. In production, Choice schools / districts will have to use the separate WI DPI WISEid system to obtain WISEids and upload or manual input to the SIS. To get a set of WISEids for testing in the UAT ‘sandbox’, submit a request and a list of WISEids for testing will be provided. SchoolId is created from the test Agency ID we create for your test school / district in the UAT ‘sandbox’ environment. This is only to the UAT ‘sandbox’ environment. You will need to go to the WISEdata Ed-Fi Credential system to get the Agency ID (this is the same system you use to get your credentials (key and secret)) that can then be used to derive the schoolId (read this blog post). To get LEA and school ids for the production environment, take a look at this blog post for more information. Retrieving students that you have already sent via a POST requires that you have sent a POST for the /studentSchoolAssociation related to that student. You can only update and retrieve students in a school/district that relate to your credentials.. educationalOrganizationId in /studentAcademicRecord is the SchoolId. It appears Ed-Fi kept this generic to allow for a district level or some other level of academic record other than school level.
https://wisconsindpi.atlassian.net/wiki/spaces/widpiedfi/pages/2294032/Data+Domains+Endpoints SCHEMASAll of the endpoints, sample data, and schema information can be found and tested via Swagger UI located here: https://uaapps.dpi.wi.gov/EdFiSwagger/
HTTP RESPONSESRESPONSE CODESA core tenet of RESTful APIs is embracing HTTP response codes to communicate status information. An API consumer should be able to inspect the HTTP response code and understand the status of its request. The following response codes are used when responding to requests. HTTP Response Code | Name | Reason(s) | 200 | OK | Returned after a successful GET, PUT, or DELETE. A PUT or DELETE response will not contain a body. | 201 | Created | Returned after a successful POST. The response from a POST will also include a Location in the Header pointing to the newly added resource. A POST response will not contain a body. | 202 | Accepted | Returned for a request that has been accepted for processing, but the processing has not yet been completed. | 204 | No Content | Returned when the server has fulfilled the request but does not return an entity body. | 304 | Not Modified | Returned when the client includes the “If-None-Match” header containing the requested resource’s last known ETag. | 400 | Bad Request | Returned if the query string for a GET or the body for a POST or PUT is malformed or missing required fields. The body of the response will contain a descriptive error message. | 401 | Unauthorized | Returned if the access token is invalid. The response will not contain a body. | 403 | Forbidden | Returned if the user to which the access token is tied does not have permissions to the requested resource. The body of the response will contain a descriptive error message. | 404 | Not Found | Returned if a resource is not found. The response will not contain a body. | 412 | Precondition Failed | Returned if an if-Match header precondition fails. | 500 | Internal Server Error | Returned if the server encountered an unexpected error during the operation.The response will not contain a body. |
ERRORS
If an error occurs on the server, a 500 (Internal Server Error) code will be returned along with the addition of a “responseStatus” property in the response body, containing the error details. For example: { "responseStatus": { "errorCode": "DataAccessException", "message": "Unable to communicate with database", "stackTrace": "…", "errors": [] } }
WISEDATA ED-FI REST SDKREQUIREMENTSThe following is an outline of the steps need to manually create the client. 1. MACHINE SETUP Download the Java SDK Download SBT and install. Just under the "Installing sbt" header download the appropriate package for your operating system. Ensure your environment can find javac. If you're on Windows set the PATH environment variable to find the jdk bin directory (e.g. C:\Program Files\Java\jdk1.7.0_51\bin). Download the ED-FI REST SDK Repository (look at the readme and its instructions) from the Double Line Partners GitHub
2. GENERATING THE REST CLIENT |