|
Fig. 1System architecture overview: Initial implementation of the SMART system (2009). |
|
Fig. 2System architecture overview: Service-oriented implementation of the SMART system (2012). |
System details | System version | SMART | Proposed | (2009) | (2012) | (2016) | Purpose | Activity recognition using Smart Homes | Reengineered based on the initial version | Reengineered with SOA implementation. | Implementation type | Standalone web application | SOA; SOAP-based; browser-based interface | SOA; REST-based; SSE and mobile application | Language (s) | C#, ASP.NET, dotNet/GWT based | Java, AJAX, JavaScript, HTML/CSS, SQL | Java and SRARQL | Main dependencies | Semantic Web (SemWeb), AJAX, Silverlight, Euler, and Pellet | PELLET (reasoning tool), Apache Jena, Mule ESB, Glassfish, JAX-WS, H2 RDBMS, AJAX | Apache Jena, Fuseki Server, JAX-RS 1.1, Jersey 2, Jersey SSE, XBee Java lib, Tyrus Web sockets, Apache Tomcat Server and Android Studio. | Interface | Browser-based | Browser-based | Mobile-based (Android application) | Portability | Single computer | One-to-many | One-to-many | Licensing | Proprietary | Open-source | Open-source |
|
Table 1Comparison between predecessors and the proposed system. |
|
Fig. 3The proposed mobile SMART system using SOA and semantic web technologies. |
|
Fig. 4Software: Breakdown of the “Sensor Utils” package. |
|
Fig. 5Hardware: Connectivity diagram of sensing devices. |
|
Fig. 6SSE mechanism for real-time message flow of sensing and inferencing results between client and web service. |
|
Fig. 7Pseudocode for executing a SPARQL query on the server endpoint using Jena API. |
|
Fig. 8Layered object properties for bucket-based structure data. |
|
Fig. 9Bucket-based approach for data structuring using object properties. |
|
Fig. 10Managing user preferences and ADL simulation mode interface. |
|
Fig. 11ADL simulation result of two possible preferences with their missing sensors to complete the activity. |
|
Fig. 12User preference management interface in action. |
|
Fig. 13Illustrating the inferencing steps taken using SPARQL query language. |
|
Fig. 14Patient’s main menu and UI of managing medicines doses. |
Activity number (#) | UAP | Sensor objects sequence | Total number of sensors | 1 | MakeIndian Tea | KitchenDoor1, KitchenCupboard1, TeaBagJar, IndianTeaSpiceJar, SugarJar, Kettle1, KitchenWaterTap1, Fridge1, MilkBottle1, | 11 | 2 | MakeCappuccino Coffee | KitchenDoor1, KitchenCupboard1, CappuccinoBagJar, SugarJar, Kettle1, KitchenWaterTap1, Fridge1, MilkBottle1, EatingSpoon1, Mug1 | 10 | 3 | MakeStawberry Juice | KitchenDoor1, KitchenCupboard1, JuicerMixerCup1, SugarJar, KitchenCupboard2, ChoppingBoard1, Knife1, Fridge1, StawberryPacket1, MilkBottle1, KitchenWaterTap1, GlassCup1, JuicerMixer1, | 13 | 4 | MakingChips AndBeans | KitchenDoor1, FridgeFreezer1, ChipsBag1, KitehenCupboard2, OvenTray1, HeinzBakedBeansCan1, KitchenWaterTap1, MicrowaveBowl1, OvenDoor1, MicrowaveDoor1, CeramicPlate1 | 11 | 5 | MakePasta | KitchenDoor1, KitchenCupboard1, PastaBag1, PastaPot1, KitchenWaterTap1, WoodCookingSpoon, PastaSauce, SaltBottle1 | 8 | 6 | TakingMedicine | KitchenCupboard1, MedicineContainer1, GlassContainer1, KitchenWaterTar1 | 4 |
|
Table 2User activity preferences with the associated total number of sensor objects. |
Scenario types | Exact no. of sensors | Extra sensors activation | Faulty/missing | TP1 | √ | × | × | TP2 | × | √ | × | TP3 | × | × | √ |
|
Table 3AR test scenario types. |
Activity number (#) | Examples of tests specifications | 1 | TP1: #1, | TP2: #1, add KitchenCupboard2 and GlasCup1. | TP3: #1, swap TeaBagJar and OvenDoor1. | 2 | TP1: #2, | TP2: #2, add KitchenCupboard2 and GlasCup1. | TP3: #2, replace Mug1 with GlassCup1. |
|
Table 4Two examples of AR test cases. |
Activity number (#) | Test type | Exp1 (ms) | Exp2 (ms) | Exp2 (ms) | Avg. (ms) | Avg. per activity number (ms) | | TP1 | 3890 | 3988 | 5127 | 4335.00 | | 1 | TP2 | 5175 | 4176 | 4802 | 4717.67 | 4472.33 | | TP3 | 4172 | 4145 | 4776 | 4364.33 | | | TP1 | 4013 | 3953 | 4439 | 4135.00 | | 2 | TP2 | 4131 | 4135 | 4725 | 4330.33 | 4287.67 | | TP3 | 4275 | 4288 | 4630 | 4397.67 | | | TP1 | 3926 | 3923 | 4353 | 4067.33 | | 3 | TP2 | 4303 | 4316 | 4571 | 4396.67 | 4410.56 | | TP3 | 5310 | 4225 | 4768 | 4767.67 | | | TP1 | 4116 | 4175 | 4452 | 4247.67 | | 4 | TP2 | 6330 | 4474 | 4695 | 5166.33 | 4636.33 | | TP3 | 4410 | 4461 | 4614 | 4495.00 | | | TP1 | 4150 | 4265 | 4409 | 4274.67 | | 5 | TP2 | 4446 | 4414 | 5919 | 4926.33 | 4584.11 | | TP3 | 4497 | 4533 | 4624 | 4551.33 | | | TP1 | 4166 | 4801 | 4271 | 4412.67 | | 6 | TP2 | 4532 | 4556 | 4563 | 4550.33 | 4473.56 | | TP3 | 4415 | 4460 | 4498 | 4457.67 | | | | | | | | 4477.43 |
|
Table 5Results showing average activity inferencing duration from the last activities recorded. |
[1] | Zhang X., Wang H., and Yu Z., Toward a smart home environment for elder people based on situation analysis, in 2010 7th International Conference on Ubiquitous Intelligence & Computing and 7th International Conference on Autonomic & Trusted Computing, 2010, pp. 7-12. |
[2] | Sterritt R. and Nugent C., Autonomic computing and ambient assisted living - extended abstract, in Engineering of Autonomic and Autonomous Systems (EASe), 2010 Seventh IEEE International Conference and Workshops on, 2010, pp. 149-151. |
[3] | Triboan D., Chen L., and Chen F., Towards a mobile assistive system using service-oriented architecture, in 2016 IEEE Symposium on Service-Oriented System Engineering Towards, 2016, pp. 187-196. |
[4] | Bohme G., Invasive Technification: Critical Essays in the Philosophy of Technology. Bloomsbury Publishing, 2012. |
[5] | Pavlic L., Hericko M., and Podgorelec V., Improving design pattern adoption with ontology-based design pattern repository, in Information Technology Interfaces, 2008. ITI 2008. 30th International Conference on, 2008, pp. 649-654. |
[6] | Ali M. and Elish M. O., A comparative literature survey of design patterns impact on software quality, in Information Science and Applications (ICISA), 2013 International Conference on, 2013, pp. 1-7. |
[7] | Zhang C., Budgen D., and Drummond S., Using a follow-on survey to investigate why use of the visitor, singleton & facade patterns is controversial, in Proceedings of the ACM—IEEE International Symposium on Empirical Software Engineering and Measurement—ESEM’12, 2012, pp. 79-88. |
[8] | Chen L., Hoey J., Nugent C. D., Cook J. D., and Yu Z., Sensor-based activity recognition, IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews, vol. 42, no. 6, pp. 790-808, 2012. |
[9] | Ameen A., Khan K. U. R., and Rani B. P., Extracting knowledge from ontology using Jena for semantic web, in 2014 International Conference for Convergence of Technology(I2CT), 2014. |
[10] | Staab S. and Rudi S., Handbook on Ontologies, 2nd Ed. Springer-Verlag, 2009. |
[11] | Culmone R., Falcioni M., Giuliodori R., Merelli E., Orru A., Quadrini M., Ciampolini P., Grossi F., and Matrella G., AAL domain ontology for event-based human activity recognition, in Mechatronic and Embedded Systems and Applications (MESA), IEEE/ASME 10th Intl Conf, 2014. |
[12] | Chen L., Nugent C., and Okeyo G., An ontology-based hybrid approach to activity modeling for smart homes, IEEE Transactions on Human-Machine Systems, vol. 44, no. 1, pp. 92-105, 2014. |
[13] | Gaaevic D., Djuric D., Devedzic V., and Selic B., Model Driven Architecture and Ontology Development. Springer-Verlag, 2006. |
[14] | Davies J., Harmelen F., and Fensel D., eds. Towards the Semantic Web: Ontology-driven Knowledge Management. John Wiley & Sons, 2002. |
[15] | Powers S., Practical RDF. O’Reilly & Associates, 2003. |
[16] | Apache, An introduction to RDF and the Jena RDF API, , 2016. |
[17] | W3C, OWL 2 web ontology language document overview, , 2012. |
[18] | DuCharme B., Learning SPARQL, 2nd Ed. O’Reilly Media, 2013. |
[19] | Pawgasame W., A survey in adaptive hybrid wireless sensor network for military operations, in 2016 Second Asian Conference on Defence Technology (ACDT), 2016, pp. 78-83. |
[20] | Hu X., Yang L., and Xiong W., A novel wireless sensor network frame for urban transportation, IEEE Internet of Things Journal, vol. 2, no. 6, pp. 586-595, 2015. |
[21] | Gaikwad P., Gabhane J. P., and Golait S. S., A survey based on smart homes system using internet-of-things, in 2015 International Conference on Computation of Power, Energy, Information and Communication (ICCPEIC), 2015, pp. 330-335. |
[22] | Khan I., Belqasmi F., Glitho R., Crespi N., Morrow M., and Polakos P., Wireless sensor network virtualization: A survey, IEEE Communications Surveys & Tutorials, vol. 18, no. 1, pp. 553-576, 2016. |
[23] | Amazon, Amazon echo, , 2016. |
[24] | Samsung, SmartThings, , 2016. |
[25] | IFTTT, Recipes on IFTTT are the easy way to automate your world, , 2016. |
[26] | Perez M. S. and Carrera E., Time synchronization in Arduino-based wireless sensor networks, IEEE Latin America Transactions, vol. 13, no. 2, pp. 455-461, 2015. |
[27] | Samsung SmartThings, SmartThings shield for Arduino, , 2016. |
[28] | Chen L., Nugent C., and Al-Bashrawi A., Semantic data management for situation-aware assistance in ambient assisted living, in Proceedings of the 11th International Conference on Information Integration and Web-based Applications & Services- IIWAS ’09, 2009. |
[29] | Chen L., Nugent C., and Rafferty J., Ontology-based activity recognition framework and services, in Proceedings of International Conference on Information Integration and Web-based Applications & Services - IIWAS ’13, 2013, pp. 463-469. |
[30] | Wang X., Wang J., Wang X., and Chen X., Energy and delay tradeoff for application offloading in mobile cloud computing, IEEE Systems Journal, 2015. doi: 10.1109/JSYST.2015.2466617. |
[31] | Martn D., Lpez de Ipia D., Alzua-Sorzabal A., Lamsfus C., and Torres-Manzanera E., A methodology and a web platform for the collaborative development of context-aware systems, Sensors, vol. 13, no. 5, p. 6032, 2013. |
[32] | Guo B., Zhang D., and Imai M., Toward a cooperative programming framework for context-aware applications, Personal and Ubiquitous Computing, vol. 15, no. 3, pp. 221-233, 2011. |
[33] | Borza P. N., Romanca M., and Delgado-Gomes V., Embedding patient remote monitoring and assistive facilities on home multimedia systems, in 2014 International Conference on Optimization of Electrical and Electronic Equipment (OPTIM), 2014, pp. 873-879. |
[34] | Kistel T., Wendlandt O., and Vandenhouten R., Using distributed feature detection for an assistive work system, in 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2014, pp. 1801-1802. |
[35] | Paola A. D., Ferraro P., Gaglio S., and Lo Re G., Autonomic behaviors in an ambient intelligence system, in 2014 IEEE Symposium on Computational Intelligence for Human-like Intelligence (IEEE SSCI 2014), 2014. |
[36] | Reichman A. and Zwiling M., The architecture of ambient assisted living system, in IEEE International Conference on Microwaves, Communications, Antennas and Electronic Systems, 2011. |
[37] | Khan A. N., Rodrguez D., Danielsson-Ojala R., Pirinen H., Kauhanen L., Salanter S., Majors J., Bjrklund S., Rautanen K., Salakoski T., et al., Smart dosing: A mobile application for tracking the medication tray-filling and dispensation processes in hospital wards, in 6th International Workshop on Intelligent Environments Supporting Healthcare and Well-being (WISHWell’14), 2014. |
[38] | Sheng Q. Z., Qiao X., Vasilakos A. V., Szabo C., Bourne S., and Xu X., Web services composition: A decade’s overview, Information Sciences, vol. 280, pp. 218-238, 2014. |
[39] | He G., Wu S., and Yao J., Application of design pattern in the JDBC programming, in the 8th International Conference on Computer Science & Education (ICCSE), 2013, pp. 1037-1040. |
[40] | Apache Jena Fuseki, , 2016. |
[41] | Hu X., Chu T., Leung V., Ngai E.C.-H., Kruchten P., and Chan H., A survey on mobile social networks: Applications, platforms, system architectures, and future research directions, IEEE Communications Surveys Tutorials, vol. 17, no. 3, pp. 1557-1581, 2014. |
[42] | Jersey, RESTful web services in Java, , 2016. |
[43] | Jersey, Server-Sent Events (SSE) support, , 2016. |
[44] | Apache, Jena ontology API, , 2016. |
[45] | Ayad M., Taher M., and Salem A., Real-time mobile cloud computing: A case study in face recognition, in 2014 28th International Conference on Advanced Information Networking and Applications Workshops (WAINA), 2014, pp. 73-78. |
[46] | Abolfazli S., Sanaei Z., Ahmed E., Gani A., and Buyya R., Cloud-based augmentation for mobile devices: Motivation, taxonomies, and open challenges, IEEE Communications Surveys and Tutorials, vol. 16, no. 1, pp. 337-368, 2014. |
[47] | Li Z. and Yap K., Context-aware discriminative vocabulary tree learning for mobile landmark recognition, Digital Signal Processing, vol. 24, pp. 124-134, 2014. |
[48] | Shimmer, Shimmer sensing, , 2015. |
[49] | Libelium, Waspmote plug & sense, , 2013. |
[50] | Care Quality Commission, About us, , 2016. |
[51] | Dentler K., Cornet R., ten Teije A., and de Keizer N., Comparison of reasoners for large ontologies in the OWL 2 EL profile, Semantic Web, vol. 2, no. 2, pp. 71-87, 2011. |
[52] | Stanford University, A free, open-source ontology editor and framework for building intelligent systems, , 2016. |
[53] | Google, Products, , 2016. |
[54] | Faludi R., Building Wireless Sensor Networks, 1st Ed. O’Reilly Media, 2010. |
[55] | Igoe T., Making Things Talk, 2nd Ed. Maker Media, Inc, 2007. |
[56] | Meditskos G., Dasiopoulou S., Vasiliki E., and Kompatsiaris I., Sp-act: A hybrid framework for complex activity recognition combining owl and sparql rules, in 2013 IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops), 2013, pp. 25-30. |
[57] | W3C, SPIN — Overview and motivation, , 2011. |
[58] | Lomotey R. K. and Deters R., Sensor data propagation in mobile hosting networks, in 2015 IEEE Symposium on Service-Oriented System Engineering (SOSE), 2015, pp. 98–106. |
[59] | Dai W. and Vyatkin V., A component-based design pattern for improving reusability of automation programs, in IECON Proceedings (Industrial Electronics Conference), 2013, pp. 4328-4333. |
[60] | Xu X., Tao Y., Wang X., and Ding X., Research on architecture of smart home networks and service platform, in 2014 5th International Conference on Digital Home (ICDH), 2014, pp. 232–236. |
[61] | Chen L., Nugent C. D., and Wang H., A knowledge-driven approach to activity recognition in smart homes, IEEE Transactions on Knowledge and Data Engineering, vol. 24, no. 6, pp. 961-974, 2012. |
[62] | Amazon Developer, Alexa — Build engaging voice experiences for your services and devices. , 2016. |
[63] | W3C, SWRL: A semantic web rule language combining OWL and RuleML, , 2004. |